Initializing and Managing Service Class Attributes at Runtime for Optimizing Deep Learning Training in Distributed Environments

Reconfigurable CLOS attributes at a subpage level dynamically adjust cacheability based on workload type, addressing inefficiencies in GPU memory allocation for deep learning, enhancing performance and scalability.

JP7589933B2Active Publication Date: 2024-11-26INTEL CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2020160931
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-12-17
Filing Date
2020-09-25
Publication Date
2024-11-26
Estimated Expiration
2040-09-25

AI Technical Summary

Technical Problem

Conventional methods for allocating buffer classes (CLOS) in graphics processing units (GPUs) lead to suboptimal performance, especially during deep neural network training with intensive communication workloads, due to static and inefficient setting of cacheability attributes.

Method used

Implementing reconfigurable CLOS attributes at a subpage level, allowing dynamic adjustment based on workload type, with methods to initialize and adjust CLOS attributes at runtime, enhancing flexibility and performance.

Benefits of technology

This approach improves performance and scalability by tailoring memory allocation to specific workloads, optimizing resource usage during deep learning training in distributed environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589933000001
    Figure 0007589933000001
  • Figure 0007589933000002
    Figure 0007589933000002
  • Figure 0007589933000003
    Figure 0007589933000003
Patent Text Reader

Abstract

To provide systems, apparatuses and methods for initialization and management of class-of-service (CLOS) attributes in runtime to optimize deep learning training in distributed environments.SOLUTION: In methods for initialization of class-of-service attributes, a technology is provided for: detecting a runtime call to a communication library, where the runtime call identifies a memory buffer; determining that a CLOS attribute is associated with the memory buffer; and issuing a driver instruction to modify the CLOS attribute in response to the runtime call.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Embodiments relate generally to class of service (CLOS) attributes and, more specifically, to initializing and managing CLOS attributes at runtime for optimizing deep learning training in distributed environments. [Background technology]

[0002] Some graphics processing units (GPUs) may allow application developers to statically set a class of service (CLOS) during buffer allocation for pages of memory that will be used during the execution of the application. The CLOS may then be used to control the cacheability of information on a per-page basis. Static setting of the CLOS during buffer allocation at the page level may be inefficient. In fact, conventional solutions may lead to suboptimal performance, especially when the application involves training of deep neural networks that have a relatively communication-intensive workload. [Brief description of the drawings]

[0003] Various advantages of those embodiments will become apparent to those skilled in the art upon reading the following specification and appended claims, and upon reference to the following drawings.

[0004] [Figure 1] FIG. 2 is an illustration of an example of multiple reconfigurable class of service attributes managed at a sub-page level according to an embodiment. [Diagram 2] 1 is a flow chart of an example method for initializing service class attributes according to an embodiment. [Diagram 3] 1 is a flow chart of an example method for adjusting class of service attributes according to an embodiment. [Figure 4] 11 is a flow chart of an example of another method for adjusting class of service attributes according to an embodiment. [Diagram 5] FIG. 1 is a signaling diagram of an example of communication between a deep learning (DL) framework, a communication library, and a driver according to an embodiment. [Figure 6] FIG. 2 is a block diagram of an example software stack according to an embodiment. [Figure 7] FIG. 1 is a block diagram of an example of a performance-enhanced computing system in accordance with an embodiment. [Figure 8] FIG. 1 is an explanatory diagram of an example of a semiconductor device according to an embodiment. [Figure 9] FIG. 2 is a block diagram of an example processor according to an embodiment. [Figure 10] FIG. 1 is a block diagram of an example multi-processor based computing system in accordance with an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Referring now to FIG. 1 , a memory page 20 (e.g., fixed-length contiguous blocks of virtual memory described by a single entry in a page table) is shown including a first address range 22 and a second address range 24. In the illustrated example, a first class of service (CLOS) attribute 26 is associated with the first address range 22 and a second CLOS 28 is associated with the second address range 24. In one embodiment, the first address range 22 is used as a first memory buffer and the second address range 24 is used as a second memory buffer. For example, when an application, such as a deep learning (DL) application, allocates a first memory buffer for a first workload, the application may also associate, dedicate, and / or assign a first CLOS attribute 26 to the first address range 22 based on the type of workload.

[0006] For example, if the DL application determines at allocation time that the first workload is expected to be a computation kernel (e.g., a software routine dedicated to matrix-matrix multiplication operations or convolution operations), then the DL application may select a relatively low value for the first CLOS attribute 26. Similarly, if the DL application determines at allocation time that the second workload is expected to be a communications kernel (e.g., a software routine dedicated to communications across multiple tiles in a multi-tile GPU package and across multiple GPU packages via scale-up links in a distributed environment), then the DL application may select a relatively high value for the second CLOS attribute 28. If the CLOS attributes 26 and 28 are proportional to cacheability, then their selected values ​​will result in less cache memory being allocated to information in the first address range 22 than the second address range 24. In this regard, it has been determined, for example, that "all-reduce" communication operations (e.g., computing gradients of a loss function on each GPU, averaging those gradients via communication between the GPUs, and updating the network model) significantly compete for the same resources as the computation kernel. In this manner, the illustrated CLOS attributes 26 and 28 can provide applications with greater flexibility in managing performance and better scalability.

[0007] In fact, the illustrated CLOS attributes 26 and 28 are reconfigurable, and the CLOS attributes 26 and 28 can further enhance efficiency, performance, and / or scalability. In this regard, the address ranges 22 and 24 may be reused (e.g., during repeated training) for multiple different workloads during the execution of the application. In this manner, the first CLOS attribute 26 may be set to a relatively high value at run-time (e.g., by instructions issued by a communication library) when the first address range 22 (e.g., a first memory buffer) is subsequently assigned to a third workload, which is a communication kernel. In this manner, even greater flexibility in terms of performance can be achieved. Furthermore, setting the CLOS attributes 26 and 28 at a level of granularity smaller than the page 20 further tailors the illustrated solution to the behavior of applications that allocate memory at the buffer level.

[0008] 2 illustrates a method 30 for initializing CLOS attributes. The method 30 may be implemented by one or more modules as a set of logical instructions stored in a machine-readable or computer-readable storage medium, such as a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), firmware, a flash memory, in configurable logic, such as, for example, programmable logic arrays (PLAs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), or in fixed-function logic hardware, such as, for example, an application specific integrated circuit (ASIC), or circuit technologies, such as complementary metal oxide semiconductor (CMOS) technology or transistor-to-transistor logic (TTL) technology, or in any combination thereof.

[0009] For example, computer program code for performing the operations shown in method 30 may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as JAVA®, SMALLTALK®, or C++, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. Additionally, the logic instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, state setting data, configuration data for integrated circuits, state information for personalizing electronic circuits and / or other structural components specific to hardware (e.g., a host processor, a central processing unit / CPU, a microcontroller, etc.).

[0010] Illustrated process block 32 provides for detecting an allocation request, where the allocation request identifies a memory buffer. In one embodiment, the allocation request identifies the memory buffer by an address range (e.g., in virtual memory). Block 34 is responsive to the allocation request to set CLOS attributes associated with the memory buffer to an initial level. Block 34 may set the CLOS attributes to a default level or may set the CLOS attributes to a level corresponding to an expected type of workload for the memory buffer. Thus, method 30 enhances performance by initializing the CLOS attributes on a per memory buffer basis (e.g., rather than on a per page basis).

[0011] 3 illustrates a method 40 for adjusting CLOS attributes. Method 40 may be implemented by one or more modules, following method 30 (FIG. 2), as a set of logical instructions stored in a machine-readable or computer-readable storage medium, such as RAM, ROM, PROM, firmware, flash memory, in configurable logic, such as, for example, a PLA, FPGA, CPLD, or in fixed function logic hardware using circuit technologies, such as, for example, ASIC, CMOS, or TTL technology, or in any combination thereof.

[0012] Illustrated processing block 42 detects a runtime call to a communications library (e.g., ccl_allreduce during a training iteration), which identifies a memory buffer. In one example, the memory buffer is identified by an address range. Block 44 determines that a CLOS attribute is associated with the memory buffer. In one embodiment, block 44 includes searching a data structure (e.g., a map) for an address range that corresponds to the memory buffer. As previously mentioned, the address range may be smaller than the size of a memory page. Block 46 issues a driver instruction to modify the CLOS attribute in response to the runtime call. In one example, if the runtime call is associated with a communications kernel, the driver instruction requests an increase in the level of the CLOS attribute. In another example, if the runtime call is associated with a compute kernel, the driver instruction requests a decrease in the level of the CLOS attribute. Block 46 may also include storing the previous value of the CLOS attribute in an appropriate memory location (e.g., a register). Thus, the illustrated method 40 enhances performance and / or scalability at least by (1) using driver instructions to modify CLOS attributes, (2) at run-time, and (3) on a per memory buffer basis.

[0013] Figure 4 illustrates another method 50 of adjusting CLOS attributes. Method 50 may be implemented by one or more modules (following method 40 of Figure 3, e.g., after augmenting CLOS attributes to support a communications kernel) as a set of logical instructions stored in a machine-readable or computer-readable storage medium, such as RAM, ROM, PROM, firmware, flash memory, etc., in configurable logic, such as a PLA, FPGA, CPLD, etc., or in fixed function logic hardware, such as using circuit technologies, such as ASIC, CMOS, or TTL technology, or in any combination thereof.

[0014] Illustrated process block 52 provides for determining whether the communications kernel has completed. If the communications kernel has not completed, the illustrated method 50 enters a wait state. Once the communications kernel has completed, block 54 returns the CLOS attributes to the initial level in response to the workload completing. In one embodiment, block 54 includes retrieving the initial level values ​​from an appropriate memory location (e.g., a register). Thus, the illustrated method 50 further enhances performance and / or scalability by providing the ability to temporarily adjust the CLOS attributes based on run-time conditions.

[0015] FIG. 5 shows a signaling diagram 60 for communication between the DL framework 62, a communication library 64 (e.g., ONEAPI collective communication library / one CCL), and a driver 66. In the illustrated example, the DL framework 62 issues a runtime call 68 (e.g., ccl_allreduce) to the communication library 64. The runtime call 68 may include buffer addresses and sizes, etc. In one embodiment, the communication library 64 issues a driver instruction (e.g., through additional middleware and / or application programming interfaces / APIs) that requests a change to the CLOS attribute for the memory region of interest. As previously mentioned, the change may be a relatively low CLOS value in the case of a compute kernel, a relatively high CLOS value in the case of a communication kernel, etc. The communication library 64 may also store the previous CLOS value for the memory region and initiate the communication operation. Once the communication is complete, the illustrated communication library 64 restores the previous CLOS value for the memory region via another driver instruction 72. In one embodiment, the communications library 64 then sends a notification 74 to the DL framework 62 that the communication is complete. Upon receiving that notification 74, the DL framework 62 may begin using the memory buffer for other purposes.

[0016] 6 illustrates a unified software stack 110, which includes a level 0 interface 112, a system software 114 below the level 0 interface 112, a system software 116 above the level 0 interface 112, and a developer interface 118. The system software 114 below the level 0 interface 112 communicates with hardware, such as a CPU (which may support, for example, scalar operations), a GPU (which may support, for example, vector operations), an AI (artificial intelligence) accelerator (which may support, for example, matrix operations), and an FPGA (which may support, for example, spatial operations). Additionally, the developer interface 118 interacts with optimized middleware and associated frameworks, which in turn support one or more optimized applications. In one embodiment, a library 120, such as (ONEAPI Collective Communication Library / ONECCL), includes the functionality of method 30 (FIG. 2), method 40 (FIG. 3), and / or method 50 (FIG. 4), which have already been described.

[0017] 7, a performance-enhanced computing system 151 is shown. The system 151 may generally be part of an electronic device / platform having a computing function (e.g., a personal digital assistant / PDA, a notebook computer, a tablet computer, a convertible tablet, a server, etc.), a communication function (e.g., a smartphone, etc.), an imaging function (e.g., a camera, a camcorder, etc.), a media playback function (e.g., a smart television / TV, etc.), a wearable function (e.g., a watch, eyewear, headwear, footwear, jewelry, etc.), an in-vehicle function (e.g., a car, a truck, a motorcycle, etc.), a robotic function (e.g., an autonomous robot, etc.), or any combination thereof. In the illustrated example, the system 151 includes a host processor 153 (e.g., a CPU with multiple cores, etc.) having a last level cache 154 (e.g., an LLC, such as a Level 3 / L3 cache, etc.), and an integrated memory controller (IMC) 155 connected to a system memory 157.

[0018] The illustrated system 151 also includes an input / output (IO) module 159 implemented together with the host processor 153 as a system on chip (SoC), and a graphics processor 161 disposed on a semiconductor die 163. The illustrated IO module 159 communicates with, for example, a display 165 (e.g., a touch screen, a liquid crystal display / LCD, a light emitting diode / LED display, etc.), a network controller 167 (e.g., wired and / or wireless), and a mass storage device 169 (e.g., a hard disk drive / HDD, an optical disk, a solid state drive / SSD, a flash memory, etc.).

[0019] In one embodiment, host processor 153, graphics processor 161, and / or IO module 159 execute program instructions 171 retrieved from system memory 157 and / or mass storage device 169 to perform one or more aspects of method 30 (FIG. 2), method 40 (FIG. 3), and / or method 50 (FIG. 4) previously described. Execution of the illustrated instructions thus causes the computing system to detect a run-time call to a communications library that identifies a memory buffer, the computing system to determine that CLOS attributes are associated with the memory buffer, and issue driver instructions to modify the CLOS attributes in response to the run-time call. In one embodiment, to determine that CLOS attributes are associated with the memory buffer, instructions 171, when executed, cause computing system 151 to search a data structure for an address range that corresponds to the memory buffer. In one example, the address range is smaller than a memory page. Thus, at least to the extent that a computing system 151 uses driver instructions to modify CLOS attributes at run-time and on a per-memory buffer basis, performance of that computing system 151 is enhanced.

[0020] FIG. 8 illustrates a semiconductor package apparatus 173. The illustrated apparatus 173 includes one or more substrates 175 (e.g., made of silicon, sapphire, gallium arsenide, etc.) and logic 177 (e.g., transistor arrays and other integrated circuit / IC components) coupled to the one or more substrates 175. The logic 177 may be implemented, at least in part, by configurable logic or fixed function logic hardware. In one example, the logic 177 implements one or more aspects of method 30 (FIG. 2), method 40 (FIG. 3), and / or method 50 (FIG. 4) previously described. In this manner, the logic 177 may detect a run-time call to a communications library that identifies a memory buffer, the logic 177 determines that CLOS attributes are associated with the memory buffer, and issues driver instructions to modify the CLOS attributes in response to the run-time call. In one embodiment, to determine that a CLOS attribute is associated with a memory buffer, logic 177 searches a data structure for an address range that corresponds to the memory buffer. In one example, the address range is smaller than a memory page. Thus, enhancing performance of semiconductor package device 173, at least to the extent that semiconductor package device 173 uses driver instructions to modify CLOS attributes at run-time and on a per-memory buffer basis.

[0021] In one example, logic 177 includes a number of transistor channel regions that are disposed (e.g., embedded) in one or more substrates 175. In this manner, the interface between logic 177 and one or more substrates 175 may not be an abrupt junction. Logic 177 may also be considered to include an epitaxial layer, which is grown on top of an initial wafer of one or more substrates 175.

[0022] FIG. 9 illustrates a processor core 200 according to one embodiment. Processor core 200 may be a core for any type of processor, such as a microprocessor, an embedded processor, a digital signal processor (DSP), a network processor, or other device for executing code. Although only one processor core 200 is illustrated in FIG. 9, a processing element may alternatively include more than the one processor core 200 illustrated in FIG. 9. Processor core 200 may be a single-threaded core, or in at least one embodiment, processor core 200 may be multi-threaded in that processor core 200 may include more than one hardware thread context (or "logic processor") per core.

[0023] FIG. 9 also illustrates memory 270 coupled to processor core 200. Memory 270 may be any of a wide variety of memories (including various layers of a memory hierarchy) known or otherwise available to those skilled in the art. Memory 270 may include one or more code 213 instructions for processor core 200 to execute, which code 213 may implement one or more aspects of method 30 (FIG. 2), method 40 (FIG. 3), and / or method 50 (FIG. 4) previously described. Processor core 200 follows a program sequence of instructions as indicated by code 213. Each instruction enters front-end portion 210 and may be processed by one or more decoders 220. Decoder 220 may generate as its output micro-operations, such as fixed-width micro-operations of a predefined format, or may generate other instructions, micro-instructions, or control signals that reflect the original code instructions. The illustrated front-end portion 210 also includes register renaming logic 225 and scheduling logic 230, which generally allocate resources and queue operations corresponding to the conversion instructions for execution.

[0024] Processor core 200 is shown to include execution logic 250 having a set of execution units 255-1 through 255-N. Some embodiments may include multiple execution units dedicated to specific functions or sets of functions. Other embodiments may include only one execution unit or one execution unit capable of performing a specific function. The illustrated execution logic 250 performs the operations specified by code instructions.

[0025] After completing execution of the operation specified by the code instruction, back-end logic 260 terminates the instruction of code 213. In one embodiment, processor core 200 allows out-of-order execution of instructions but requires in-order termination of instructions. Termination logic 265 may take a variety of forms known to those skilled in the art (e.g., a reorder buffer, etc.). In this manner, processor core 200 translates at least the form of the outputs generated by the decoders, the hardware registers and tables utilized by register renaming logic 225, and any registers modified by execution logic 250 (not shown) during execution of code 213.

[0026] 9, a processing element may include other elements located on a chip with processor core 200. For example, a processing element may include memory control logic along with processor core 200. A processing element may include I / O control logic and / or I / O control logic integrated with the memory control logic. A processing element may also include one or more caches.

[0027] Referring now to Figure 10, a block diagram of an embodiment of a computing system 1000 is shown in accordance with one embodiment. In Figure 10, a multiprocessor system 1000 is shown that includes a first processing element 1070 and a second processing element 1080. Although two processing elements 1070 and 1080 are shown, it should be understood that an embodiment of the system 1000 may also include only one such processing element.

[0028] System 1000 is illustrated as a point-to-point interconnect system, where a first processing element 1070 and a second processing element 1080 are connected by a point-to-point interconnect 1050. It should be understood that any or all of the interconnects illustrated in Figure 10 may be implemented as multi-drop buses rather than point-to-point interconnects.

[0029] 10, each of the processing elements 1070 and 1080 may be a multi-core processor that includes a first processor core and a second processor core (i.e., processor cores 1074a and 1074b, and processor cores 1084a and 1084b). Such cores 1074a, 1074b, 1084a, and 1084b may be configured to execute instruction codes in a manner similar to that described above in connection with FIG.

[0030] Each processing element 1070 and 1080 may include at least one shared cache 1896a and 1896b. The shared caches 1896a and 1896b may store data (e.g., instructions) utilized by one or more components of the processor, such as the cores 1074a, 1074b, 1084a, and 1084b, respectively. For example, the shared caches 1896a and 1896b may locally cache data stored in the memories 1032 and 1034 for faster access by the components of the processor. In one or more embodiments, the shared caches 1896a and 1896b may include one or more mid-level caches, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level cache, a last level cache (LLC), and / or combinations thereof.

[0031] While shown using only two processing elements 1070 and 1080, it should be understood that the scope of the embodiments is not so limited. In other embodiments, one or more additional processing elements may be present in a given processor. Alternatively, one or more of the processing elements 1070 and 1080 may be elements other than a processor, such as, for example, an accelerator or a field programmable gate array. For example, the one or more additional processing elements may include an additional processor that is the same as the first processor 1070, one or more additional processors that are heterogeneous or asymmetric processors relative to the first processor 1070, an accelerator (e.g., a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array, or any other processing element. There may be various differences between the processing elements 1070 and 1080 with respect to a range of criteria of merit, including architecture, microarchitecture, thermal, and power consumption characteristics, etc. These differences may manifest substantially as asymmetries and non-uniformities between the processing elements 1070 and 1080. For at least one embodiment, the various processing elements 1070 and 1080 may reside within the same die package.

[0032] The first processing element 1070 may further include memory controller logic (MC) 1072 and point-to-point (PP) interfaces 1076 and 1078. Similarly, the second processing element 1080 may include MC 1082 and PP interfaces 1086 and 1088. As shown in FIG. 10, the MCs 1072 and 1082 couple the processors to respective memories, i.e., memories 1032 and 1034, which may be part of a main memory locally associated with the respective processors. Although the MCs 1072 and 1082 are shown as being integrated within the processing elements 1070 and 1080, in alternative embodiments, the MC logic may be separate logic outside the processing elements 1070 and 1080, rather than being integrated within the processing elements 1070 and 1080.

[0033] The first processing element 1070 and the second processing element 1080 may be coupled to the I / O subsystem 1090 via PP interconnects 1076 and 1086, respectively. As shown in FIG. 10, the I / O subsystem 1090 includes PP interfaces 1094 and 1098. Additionally, the I / O subsystem 1090 includes an interface 1092 for coupling the high performance graphics engine 1038 to the I / O subsystem 1090. In one embodiment, the bus 1049 may be used to couple the graphics engine 1038 to the I / O subsystem 1090. Alternatively, a point-to-point interconnect may couple these components.

[0034] I / O subsystem 1090 may in turn be coupled to a first bus 1016 via an interface 1096. In one embodiment, first bus 1016 may be a bus such as a Peripheral Component Interconnect (PCI) bus, or a PCI Express bus or other third generation I / O interconnect bus, although the scope of the embodiment is not so limited.

[0035] As shown in FIG. 10, various I / O devices 1014 (e.g., biometric scanner, speaker, camera, sensors, etc.) may be coupled to a first bus 1016 along with a bus bridge 1018 that may couple the first bus 1016 to a second bus 1020. In one embodiment, the second bus 1020 may be a low pin count bus. In one embodiment, various devices may be coupled to the second bus 1020, including, for example, a keyboard / mouse 1012, one or more communication devices 1026, and a data storage unit 1019, such as a disk drive or other mass storage device, that may include code 1030. The illustrated code 1030 may implement one or more aspects of method 30 (FIG. 2), method 40 (FIG. 3), and / or method 50 (FIG. 4) previously described. Additionally, an audio I / O 1024 may be coupled to a second bus 1020 , and a battery 1010 may provide power to the computing system 1000 .

[0036] It should be noted that numerous other embodiments are possible. For example, instead of the point-to-point architecture of Figure 10, the system may implement a multi-drop bus or other such communication topology. Also, the elements of Figure 10 may alternatively be split using more or fewer integrated chips than shown in Figure 10.

[0037] Additional notes and examples:

[0038] Example 1 is a performance enhancing computing system, comprising: A network controller; a processor coupled to the network controller; and a memory coupled to the processor, the memory including a set of executable program instructions that, when executed by the processor, cause the performance-enhanced computing system to: Detecting a runtime call to a communications library, the runtime call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with the memory buffer; and In response to the runtime call, the performance enhancing computing system issues driver instructions to modify the CLOS attributes.

[0039] Example 2 includes the performance-enhanced computing system of example 1, wherein the executable program instructions, when executed, cause the performance-enhanced computing system to search a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page, to determine that the CLOS attribute is associated with the memory buffer.

[0040] Example 3 includes the performance-enhanced computing system of example 1, wherein the driver instructions are configured to request an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

[0041] Example 4 includes the performance-enhanced computing system of example 3, where execution of the executable program instructions further causes the performance-enhanced computing system to return the CLOS attribute to an initial level in response to completion of the communications kernel.

[0042] Example 5 includes the performance-enhanced computing system of example 1, wherein the driver instructions are configured to request a reduction in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

[0043] Example 6 further comprises the step of, when the executable program instructions are executed, causing the performance-enhanced computing system to: detecting an allocation request, the allocation request identifying the memory buffer; and In response to the allocation request, the performance enhancing computing system of any one of Examples 1 to 5 causes the CLOS attribute to be set to an initial level.

[0044] Example 7 is a semiconductor device, one or more substrates; and logic coupled to the one or more substrates, the logic being implemented, at least in part, by one or more of configurable logic or fixed function hardware logic, the logic coupled to the one or more substrates comprising: Detecting a runtime call to a communications library, the runtime call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with the memory buffer; and A semiconductor device is included that is responsive to the runtime call to issue driver instructions to modify the CLOS attributes.

[0045] Example 8 includes the semiconductor device of example 7, wherein to determine that the CLOS attribute is associated with the memory buffer, the logic coupled to the one or more substrates is configured to search a data structure for an address range corresponding to the memory buffer, the address range being smaller than a memory page.

[0046] Example 9 includes the semiconductor device of example 7, wherein the driver instruction is configured to request an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

[0047] Example 10 includes the semiconductor device of example 9, wherein the logic coupled to the one or more substrates is configured to, in response to completion of the communications kernel, return the CLOS attribute to an initial level.

[0048] Example 11 includes the semiconductor device of example 7, wherein the driver instruction is configured to request a reduction in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

[0049] Example 12 is a cross-sectional view of the logic coupled to the one or more substrates, the logic comprising: detecting an allocation request, the allocation request identifying the memory buffer; and 12. The semiconductor device according to any one of Examples 7 to 11, configured to set the CLOS attribute to an initial level in response to the allocation request.

[0050] Example 13 is at least one computer-readable storage medium including a set of executable program instructions that, when executed by a computing system, cause the computing system to: Detecting a runtime call to a communications library, the runtime call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with the memory buffer; and and at least one computer readable storage medium configured to: issue driver instructions to modify said CLOS attributes in response to said runtime calls.

[0051] Example 14 includes at least one computer-readable storage medium of Example 13, wherein the executable program instructions, when executed, cause the computing system to search a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page, to determine that the CLOS attribute is associated with the memory buffer.

[0052] Example 15 includes at least one computer-readable storage medium of example 13, wherein if the runtime call is associated with a communications kernel, the driver instructions are configured to request an increase in the level of the CLOS attribute.

[0053] Example 16 includes at least one computer-readable storage medium of Example 15, wherein the executable program instructions, when executed, further cause the computing system to return the CLOS attribute to an initial level in response to completion of the communications kernel.

[0054] Example 17 includes at least one computer-readable storage medium of example 13, wherein the driver instructions are configured to request a reduction in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

[0055] Example 18 further comprises the computing system, when the executable program instructions are executed, further comprising: detecting an allocation request, the allocation request identifying the memory buffer; and 18. In response to the allocation request, the CLOS attribute is set to an initial level.

[0056] Example 19 is a method of operating an enhanced performance computing system, the method comprising: detecting a run-time call to a communications library, the run-time call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with the memory buffer; and c) in response to the runtime call, issuing driver instructions to modify the CLOS attributes.

[0057] Example 20 includes the method of example 19, wherein determining that the CLOS attribute is associated with the memory buffer includes searching a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page.

[0058] Example 21 includes the method of example 19, wherein the driver instruction requests an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

[0059] Example 22 includes the method of example 21, further comprising returning the CLOS attribute to an initial level in response to completion of the communications kernel.

[0060] Example 23 includes the method of example 19, wherein the driver instruction requests a decrease in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

[0061] Example 24 is detecting an allocation request, the allocation request identifying the memory buffer; 24. The method of any one of Examples 19-23, further comprising: in response to the allocation request, setting the CLOS attribute to an initial level.

[0062] Example 25 includes a means for carrying out the method of any one of Examples 19-24.

[0063] In this manner, the techniques described herein use instruction-based setting of CLOS attributes to allow communication libraries to select which buffers have these CLOS attributes. The techniques also allow the communication libraries to increase or decrease CLOS priority based on runtime behavior. Thus, a mechanism for tradeoff between computational and communication performance is achieved. Indeed, the ability to fine-tune training for DL ​​workloads can be particularly advantageous.

[0064] The embodiments are applicable for use with all types of semiconductor integrated circuit ("IC") chips. Examples of these IC chips include, but are not limited to, processors, controllers, chipset components, programmable logic arrays (PLAs), memory chips, network chips, systems on chips (SoCs), and SSD / NAND controller ASICs. Additionally, in some of the figures, signal conductor traces are represented by lines. Some of the lines may be different to indicate more component signal paths, may have a numeric label to indicate multiple component signal paths, and / or may have arrows at one or more ends to indicate the direction of the primary information flow. However, this should not be construed as limiting. Rather, such additional details may be used in connection with one or more exemplary embodiments to facilitate easier understanding of a circuit. Any represented signal lines, whether or not carrying additional information, may in fact include one or more signals that may propagate in multiple directions and may be implemented by any suitable type of signaling scheme, such as digital or analog lines implemented by differential pairs, fiber optic lines, and / or single-ended lines.

[0065] Although exemplary sizes / models / values / ranges may be given, the embodiments are not limited to the same. It is expected that as manufacturing techniques (e.g., photolithography) mature over time, it will be possible to manufacture devices with smaller sizes. In addition, well-known power / ground connections to IC chips and other components may or may not be shown in the figures for ease of illustration and explanation, and so as not to obscure the specific aspects of the embodiments. Furthermore, configurations may also be shown in block diagram form in consideration of the fact that details regarding the implementation of such block diagram configurations are highly dependent on the computing system in which the embodiments are implemented, i.e., such details should be within the scope of the creative activity of a person skilled in the art. Where specific details (e.g., circuits, etc.) are described to describe exemplary embodiments, it should be apparent to one skilled in the art that the embodiments can be realized without or with variations of these specific details. In this manner, the description should be considered as exemplary and not limiting.

[0066] The term "coupled" may be used herein to refer to any type of direct or indirect relationship between components of interest and may apply to electrical, mechanical, fluid, optical, electromagnetic, electromechanical, or other connections. In addition, terms such as "first," "second," etc. may be used for ease of description only and may not have a particular temporal or chronological significance unless otherwise indicated.

[0067] As used in this application and in the claims, a list of items combined by the words "one or more of" may mean any combination of those listed words. For example, the phrase "one or more of A, B, or C" may mean A, B, C, A and B, A and C, B and C, or A, B, and C.

[0068] Those skilled in the art will appreciate from the above description that the broad techniques of the embodiments may be implemented in a variety of forms. Thus, while the embodiments have been described with reference to specific examples of the embodiments, the actual scope of the embodiments should not be so limited, as other modifications will be apparent to those of skill in the art upon review of the drawings, specification, and claims that follow.

Claims

1. 1. A computing system comprising: A network controller; a processor coupled to the network controller; and a memory coupled to the processor, the memory including a set of executable program instructions that, when executed by the processor, cause the computing system to: Detecting a runtime call to a communications library, the runtime call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with said memory buffer; and in response to the runtime call, issuing driver instructions to modify the CLOS attributes. Computing system.

2. 2. The computing system of claim 1, wherein the executable program instructions, when executed, cause the computing system to search a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page, to determine that the CLOS attribute is associated with the memory buffer.

3. The computing system of claim 1 , wherein the driver instructions are configured to request an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

4. 4. The computing system of claim 3, wherein the executable program instructions, when executed, further cause the computing system to return the CLOS attribute to an initial level in response to completion of the communications kernel.

5. The computing system of claim 1 , wherein the driver instructions are configured to request a reduction in the level of the CLOS attributes if the runtime call is associated with a compute kernel.

6. The executable program instructions, when executed, further cause the computing system to: detecting an allocation request, the allocation request identifying the memory buffer; and 6. A computing system as claimed in claim 1, further comprising: in response to the allocation request, the CLOS attribute is set to an initial level.

7. A semiconductor device comprising: one or more substrates; and logic coupled to the one or more substrates, the logic being implemented, at least in part, by one or more of configurable logic or fixed function hardware logic, the logic coupled to the one or more substrates comprising: Detecting a runtime call to a communications library, the runtime call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with said memory buffer; and in response to the runtime call, issuing driver instructions to modify the CLOS attributes; Semiconductor device.

8. 8. The semiconductor device of claim 7, wherein to determine that the CLOS attribute is associated with the memory buffer, the logic coupled to the one or more substrates is configured to search a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page.

9. The semiconductor device of claim 7 , wherein the driver instructions are configured to request an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

10. 10. The semiconductor device of claim 9, wherein the logic coupled to the one or more substrates is configured to, in response to completion of the communications kernel, return the CLOS attribute to an initial level.

11. The semiconductor device of claim 7 , wherein the driver instructions are configured to request a reduction in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

12. The logic coupled to the one or more substrates includes: detecting an allocation request, the allocation request identifying the memory buffer; and 12. The semiconductor device according to claim 7, configured to set the CLOS attribute to an initial level in response to the allocation request.

13. 1. A computer program comprising a set of executable program instructions, the executable program instructions, when executed by a computing system, causing the computing system to: Detecting a runtime call to a communications library, the runtime call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with said memory buffer; and in response to the runtime call, issuing driver instructions to modify the CLOS attributes. Computer program.

14. 14. The computer program product of claim 13, wherein the executable program instructions, when executed, cause the computing system to search a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page, to determine that the CLOS attribute is associated with the memory buffer.

15. 14. The computer program product of claim 13, wherein the driver instructions are configured to request an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

16. 16. The computer program product of claim 15, wherein the executable program instructions, when executed, further cause the computing system to, in response to completion of the communications kernel, return the CLOS attribute to an initial level.

17. 14. The computer program product of claim 13, wherein the driver instructions are configured to request a reduction in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

18. The executable program instructions, when executed, further cause the computing system to: detecting an allocation request, the allocation request identifying the memory buffer; and 18. A computer program product as claimed in any one of claims 13 to 17, causing, in response to the allocation request, to set the CLOS attribute to an initial level.

19. 1. A method comprising: detecting a run-time call to a communications library, the run-time call identifying a memory buffer; determining that a class of service (CLOS) attribute is associated with said memory buffer; and in response to the runtime call, issuing driver instructions to modify the CLOS attributes. method.

20. 20. The method of claim 19, wherein determining that the CLOS attribute is associated with the memory buffer comprises searching a data structure for an address range that corresponds to the memory buffer, the address range being smaller than a memory page.

21. 20. The method of claim 19, wherein the driver instruction requests an increase in the level of the CLOS attribute if the runtime call is associated with a communications kernel.

22. 22. The method of claim 21, further comprising the step of returning the CLOS attributes to an initial level in response to completion of the communications kernel.

23. 20. The method of claim 19, wherein the driver instruction requests a decrease in the level of the CLOS attribute if the runtime call is associated with a compute kernel.

24. detecting an allocation request, the allocation request identifying the memory buffer; 24. The method of claim 19, further comprising the step of: in response to the allocation request, setting the CLOS attribute to an initial level.

25. Apparatus comprising means for carrying out the method according to any one of claims 19 to 23.

26. A computer readable storage medium storing a computer program according to any one of claims 13 to 18.

Citation Information

Patent Citations

  • Packet transfer apparatus, scheduler, data transmission apparatus, and packet transfer method

    JP2003318964A

  • System and method for scheduling packet transmission on a client device

    JP2015511449A

  • Scalable reconfigurable router

    US20080123654A1

  • Network switching architecture utilizing cell based and packet based per class-of-service head-of-line blocking prevention

    US6154446A