Network device and system for synchronizing data, and method for operating CXL switching device

Abstract

A network device, a system, and a method of operating a CXL switching device for synchronizing data are provided. Various example embodiments may include a method of operating a network device for synchronizing data, a non-transitory computer-readable medium including computer-readable instructions for operating a network device, a system including a network device, and / or a compute fast link (CXL) switching device. A CXL-based system includes: a plurality of CXL processing devices configured to perform a matrix multiplication calculation based on input vector data and sub-matrices, and output at least one interrupt signal and at least one data packet based on a result of the matrix multiplication calculation, the at least one data packet comprises output vector data and characteristic data associated with the output vector data; and a CXL switching device configured to: synchronize the output vector data, the synchronization including performing a calculation operation on the output vector data based on the interrupt signal and the data packet; and providing the synchronized vector data to a plurality of CXL processing devices.

Description

Technical Field

[0001] Various example embodiments of the inventive concepts relate to electronic devices, and more particularly, to a network device, a system, a non-transitory computer-readable medium, and / or a method of operating a Compute Express Link (CXL) switch device for synchronizing data. Background Art

[0002] With the development of technologies such as artificial intelligence (AI), big data, and edge computing, there is an increasing need to process large amounts of data quickly on devices. High-bandwidth applications that perform complex calculations expect and / or require faster data processing and / or more efficient memory access. For example, in very large artificial intelligence models such as large language models (LLMs), a large number of parameters are processed for docking. For this purpose, a technology is being developed that divides the weight matrix and stores it in processing devices such as multiple GPUs and / or FPGA devices, and each device processes data in parallel. At this time, since the data and / or results are calculated separately in each device based on partial information, results and / or data based on overall information are required, so a data synchronization process for the overall results and / or data is desired and / or necessary. Typically, in the case of a synchronization process, partial data calculated by each device is sent to a central device (and / or desired device), and the central device synchronizes partial data based on overall information, and resends synchronized data to each device. However, this synchronization process can cause bottlenecks and / or congestion between devices, thereby increasing computational latency. Summary of the invention

[0003] According to at least one example embodiment of the inventive concept, a system based on a compute fast link (CXL) is provided, the system comprising: a plurality of CXL processing devices, each of the CXL processing devices being configured to perform a matrix multiplication calculation based on input vector data and a partial matrix, and output at least one interrupt signal and at least one data packet based on a result of the matrix multiplication calculation, each of the at least one data packet comprising output vector data and characteristic data associated with the output vector data; and a CXL switching device, the CXL switching device being configured to: synchronize the output vector data to generate synchronized vector data, the synchronization comprising performing a calculation operation on the output vector data based on the at least one interrupt signal and the at least one data packet; and provide the synchronized vector data to the plurality of CXL processing devices.

[0004] According to at least one example embodiment of the inventive concept, a method of operating a compute fast link (CXL) switch is provided, the method comprising: receiving a plurality of data packets and at least one interrupt signal from a plurality of CXL processing devices, wherein each of the plurality of data packets comprises vector data and characteristic data associated with the vector data; synchronizing the vector data, the synchronizing comprising performing a computational operation on the vector data based on the plurality of data packets and the at least one interrupt signal; and outputting the synchronized vector data to the plurality of CXL processing devices.

[0005] According to at least one example embodiment of the inventive concept, a network device based on a compute fast link (CXL) is provided, the network device comprising: a memory, the memory being configured to store vector data of multiple data packets received from multiple processing devices; and a processing circuit, the processing circuit being configured to: in response to multiple interrupt signals received from the multiple processing devices, generate at least one instruction signal based on characteristic data in the multiple data packets, and store the at least one instruction signal in the memory; determine a computing operation type based on the at least one instruction signal stored in the memory; synchronize the vector data stored in the memory, the synchronization comprising performing the computing operation on the vector data based on the determined computing operation type; and output the synchronized vector data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various exemplary embodiments of the inventive concept will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0007] Figure 1 is a block diagram illustrating a system according to at least one example embodiment;

[0008] Figure 2A , Figure 2B and Figure 2C is a diagram for describing the operation of a system according to at least one example embodiment;

[0009] Figure 3 is a block diagram illustrating control logic according to at least one example embodiment;

[0010] Figure 4 is a block diagram illustrating a memory according to at least one example embodiment;

[0011] Figure 5 is a block diagram illustrating computing logic according to at least one example embodiment;

[0012] Figure 6 is a block diagram illustrating a CXL processing apparatus according to at least one example embodiment;

[0013] Fig. 7A and Figure 7B is a diagram for describing a data flow according to a comparative example and at least one example embodiment;

[0014] Figure 8 is a flowchart of a method of operating a system according to at least one example embodiment;

[0015] Fig. 9 Is used to describe Figure 8 A flowchart of at least one example embodiment of operation S20;

[0016] Fig.10 Is used to describe Fig. 9 A flowchart of at least one example embodiment of operation S220 of ; and

[0017] Fig.11 Is used to describe Fig. 9 0046] A flowchart of at least one example embodiment of operation S230 of FIG. DETAILED DESCRIPTION

[0018] Figure 1 is a block diagram illustrating a system according to at least one example embodiment.

[0019] refer to Figure 1, the system 1 may support the Compute Express Link (CXL) protocol, but is not limited thereto. CXL may be used as a corresponding protocol for other protocols (such as fabric-based Non-Volatile Memory Express (NVMeoF), etc.) that may be used for configurability (e.g., configuration) of at least one remote input / output (I / O) device. As used herein, the term "composable" may refer to the property that a given device (e.g., a cache coherence-enabled device in a particular cluster, etc.) is able to request and / or obtain resources (e.g., memory, computing and / or network resources, etc.) from another part of the network (e.g., at least another cache coherence-enabled device in a second cluster, etc.) to run at least a portion of a workload (e.g., computer processing, mathematical processing, neural network processing, LLM processing, AI processing, etc.). According to some example embodiments, the term "composability" may include using a flexible pool of physical and virtual computing, storage, and / or fabric resources, etc., in any suitable configuration to run any application and / or workload. CXL is an open industry standard for communication based on the Peripheral Component Interconnect Express (PCIe) 5.0 protocol, which can provide fixed, relatively short packet sizes, thereby providing relatively high bandwidth and / or relatively low fixed latency. Therefore, CXL can support cache coherence, and CXL can be well suited for creating connections to memory (e.g., at least one memory device, etc.). CXL can also be used by at least one server to provide a connection between at least one host (e.g., at least one host device, etc.) and an accelerator, a memory device, and / or a network interface circuit (e.g., a "network interface controller" and / or a network interface card (NIC), etc.). Cache coherence protocols such as CXL can be used for heterogeneous processing in, for example, scalar, vector, and / or buffered memory systems, but are not limited thereto. CXL can be used to provide a cache coherent interface by utilizing channels, retimers, the PHY layer of the system, logical aspects of the interface, and / or protocols from the PCIe 5.0 protocol. The CXL transaction layer may include three multiplexed sub-protocols that operate simultaneously on a single link and may be referred to as CXL.io, CXL.cache, and CXL.memory. CXL.io may include I / O semantics that may be similar to PCIe. CXL.cache may include cache semantics, CXL.memory may include memory semantics, and both cache semantics and memory semantics may be optional.Similar to PCIe, CXL supports: (i) divisible native widths of x16, x8, and x4; (ii) data rates of 8GT / s, 16GT / s, and 32GT / s that can be degraded to 128b / 130b; (iii) 300W (e.g., a x16 connector can support 75W); and (iv) plug-and-play. To support plug-and-play, a PCIe and / or CXL device link can start training Gen1 PCIe, negotiate CXL, and initiate CXL transactions after completing Gen 1-5 training, etc.

[0020] The system 1 may include a CXL host 10 (e.g., a CXL host device, etc.), a CXL switch 100, and / or first to nth CXL processing devices 110_1, 110_2, ..., and 110_n, but example embodiments are not limited thereto, and for example, the system 1 may include a greater or lesser number of constituent devices. n may be an integer equal to or greater than 2.

[0021] The CXL host 10 may process data using a processing circuit (e.g., a central processing unit (CPU), an application processor (AP), and / or a system on chip (SoC), etc.). The CXL host 10 may run an operating system (OS) and / or various applications (e.g., software applications, etc.). The CXL host 10 may be connected to a host memory. The CXL host 10 may include a physical layer, a multi-protocol multiplexer, an interface circuit, a coherence / cache circuit, a bus circuit, at least one core (e.g., a processor core, etc.), and / or at least one input / output device, etc., but is not limited thereto, and for example, may include a greater or lesser number of constituent elements. The CXL host 10 is connected to the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n through at least one CXL interface, and may generally control the operations of the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n. The CXL interface is an interface capable of reducing overhead and latency of a host device and a semiconductor device and allowing space of a host memory and a device memory to be shared in a heterogeneous computing environment in which the CXL host 10 and the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n work together due to rapid innovation of special workloads such as data compression and encryption and artificial intelligence (AI). The CXL host 10 and the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n can maintain and / or improve memory consistency at a high bandwidth and / or a very high bandwidth through the CXL interface. The CXL interface includes at least three sub-protocols, such as CXL.io, CXL.cache, and CXL.mem. For example, CXL.io uses a PCIe interface and is used to search for devices in a system, manage interrupts, provide access through registers, process initialization, and / or process signal errors, etc. CXL.cache can be used when a computing device such as an accelerator included in a semiconductor device accesses a host memory of a host device, etc. CXL.mem may be used by a host device to access a device memory or the like included in a semiconductor device.

[0022] The CXL switch 100 may synchronize the output vector data by performing calculations on the output vector data based on interrupt signals and / or data packets (e.g., data packets). The CXL switch 100 may provide the synchronized vector data to the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n. The CXL switch 100 according to at least one example embodiment of the inventive concept may be referred to as a CXL switching device and / or a CXL-based network device, etc. The use of a CXL connection to at least one memory pool may provide various advantages and / or technical advantages in a system including, for example, a plurality of servers connected to each other via a network, but example embodiments are not limited thereto. For example, the CXL switch 100 may have additional functions in addition to providing a packet switching function for CXL packets. The CXL switch 100 may be used to connect a memory pool to one or more CXL hosts 10 and / or one or more network interface circuits. Accordingly, (i) a memory set may include various types of memory with different characteristics, (ii) the CXL switch 100 may virtualize the memory set and enable data of different characteristics (e.g., access frequency, etc.) to be stored in the appropriate type of memory, and / or (iii) the CXL switch 100 may support remote direct memory access (RDMA) so that RDMA, etc., may be performed with little and / or no involvement of the processing circuitry of the server. The term "virtualizing memory" refers to performing memory address translation between the processing circuitry and the memory, for example, translating a virtual memory address associated with a software application such as an operating system, etc. into a physical address of (one or more) memory devices. Additionally, the CXL switch 100 may: (i) support isolation between memory and accelerators through a single-stage switch; (ii) support offline and online switching of resources between domains and implement cross-domain time multiplexing when requested; and / or (iii) support virtualization of downstream ports, etc. CXL may be used to implement a memory set that enables one-to-many switching and many-to-one switching when an aggregate device is partitioned into multiple logical devices each having a logical device identifier (LD-ID). For example, (i) CXL may connect multiple root ports to one endpoint, (ii) connect one root port to multiple endpoints, and / or (iii) connect multiple root ports to multiple endpoints, etc. According to some example embodiments, a physical device may be partitioned into multiple logical devices, each of which is visible to an initiator. A device may have one physical function (PF) and multiple (e.g., 16, etc.) separate logical devices. According to some example embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., up to 16), and there may also be one control partition (which may be a PF for controlling the device), but example embodiments are not limited thereto.The CXL switch 100 may include a plurality of input / output ports configured to connect to a network and / or fabric. For example, each input / output port of the CXL switch 100 may support a CXL interface and implement a CXL protocol, but is not limited thereto.

[0023] According to some example embodiments, the CXL switch 100 may include a control logic 101, a memory 102, and / or a computing logic 103, but the example embodiments are not limited thereto. According to some example embodiments, one or more of the control logic 101, the memory 102, the computing logic 103, etc. may be implemented as a processing circuit. The processing circuit may include: hardware or a hardware circuit including a logic circuit; a hardware / software combination, such as a processor running software and / or firmware; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc., but is not limited thereto.

[0024] The control logic 101 may store at least one instruction signal in the memory 102 based on the characteristic data in response to at least one interrupt signal. For example, the control logic 101 may store instruction signals in the memory based on the characteristic data in the plurality of data packets in response to a plurality of interrupt signals received from the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n, but is not limited thereto.

[0025] The memory 102 may be implemented as a volatile memory, but is not limited thereto, and, for example, may be implemented as a nonvolatile memory. The volatile memory may include, for example, a static random access memory (SRAM), but is not limited thereto. In another example, the volatile memory may include a dynamic random access memory (DRAM), a mobile DRAM, a double data rate synchronous dynamic random access memory (DDR SDRAM), a low power DDR (LPDDR) SDRAM, a graphic DDR (GDDR) SDRAM, a Rambus dynamic random access memory (RDRAM), etc. The memory 102 may temporarily store output vector data provided from the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n. The memory 102 may temporarily store at least one instruction signal provided from the control logic 101. Likewise, the memory 102 may store vector data of a plurality of data packets received from the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n.

[0026] The calculation logic 103 may determine a calculation operation based on at least one instruction signal stored in the memory 102. The calculation logic 103 may synchronize the vector data stored in the memory 102 by performing at least one calculation operation on the vector data. The calculation logic 103 may generate synchronized vector data according to the calculation operation. The calculation logic 103 may store the synchronized vector data in the memory 102.

[0027] The first to nth CXL processing devices 110_1, 110_2, ..., and 110_n may be connected below the CXL switch 100, so the plurality of CXL processing devices 110_1, 110_2, ..., and 110_n may be configured as a memory pool. Each of the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n may perform, for example, matrix multiplication calculations of input vector data and sub-matrices, but example embodiments are not limited thereto, and other forms of mathematical calculations may be performed on the input data instead. Each of the first to nth CXL processing devices 110_1, 110_2, ..., and 110_n may output at least one data packet and at least one interrupt signal to the CXL switch 100. The data packet (e.g., data packet) may include output vector data and / or characteristic data, but example embodiments are not limited thereto. A unit of data transmitted per clock cycle may be referred to as a data packet. A data packet (e.g., data packet, etc.) according to the CXL specification may also be referred to as a flow control unit (flit). The data packet may include a protocol ID field, a plurality of time slots and / or a CRC field, etc., but is not limited thereto. The protocol ID may be information identifying a plurality of protocols supported by a link and / or a connection (e.g., CXL). The time slot may be an area in a data packet containing at least one message. The message may include, for example, a valid field, an operation code opcode field, an address ADDR field, and / or a reserved RSVD field, etc., but is not limited thereto. The number of fields included in the message, the size of the field, and / or the type of the field may vary depending on the protocol. Each field included in the message may include at least one bit of data and / or information, etc. The valid field may include 1 bit indicating whether the message is a valid message or an invalid message and / or used to determine whether the message is a valid message or an invalid message, etc. The opcode field may include a plurality of bits defining an operation corresponding to the message. The ADDR field may include a plurality of bits representing an address (e.g., a memory address, etc.) associated with the opcode field. The RSVD field may be an area that may include additional information. Therefore, information newly added to the message by the protocol may be included in the RSVD field. The CRC field may include one or more bits for transmission error detection.

[0028] Figure 2A , Figure 2B and Figure 2C is a diagram for describing the operation of a system according to at least one example embodiment. Figure 2A , Figure 2B and Figure 2C In the description, for convenience of explanation, it is assumed that the plurality of CXL processing devices include a first CXL processing device 110_1 and a second CXL processing device 110_2 , but example embodiments are not limited thereto, and for example, there may be a greater number of CXL processing devices.

[0029] refer to Figure 2A , each of the plurality of CXL processing devices may store information about the submatrix in advance, but is not limited thereto, and, for example, the plurality of CXL processing devices may receive the submatrix, etc. For example, the first CXL processing device 110_1 may store information about the first submatrix 211, and the second CXL processing device 110_2 may store information about the second submatrix 212. The information about the submatrix may be stored in at least one register provided in each of the plurality of CXL processing devices. According to some example embodiments, the submatrix stored in each of the plurality of CXL processing devices may correspond to a portion of a weight matrix of an AI model, but example embodiments are not limited thereto. For example, the first submatrix 211 and the second submatrix 212 may be matrices divided by columns in the weight matrix of the AI ​​model. However, example embodiments of the inventive concept are not limited thereto. The AI ​​model may include, for example, various types of models, including a large language model (LLM) (such as GPT-3 and / or GPT-4), a convolutional neural network (CNN), and / or a region proposal network (RPN), etc. According to another example embodiment, the submatrices stored in the plurality of CXL processing devices may be identical to each other. For example, the first sub-matrix 211 and the second sub-matrix 212 may be identical to each other.

[0030] Each of the plurality of CXL processing devices may perform operations on the stored sub-matrix and the received input vector data, such as matrix multiplication calculation of the sub-matrix and the input vector data, but is not limited thereto. For example, the first CXL processing device 110_1 may perform a matrix multiplication calculation 231 of the first sub-matrix 211 and the first input vector data 221. The second CXL processing device 110_2 may perform a matrix multiplication calculation 232 of the second sub-matrix 212 and the second input vector data 222. The input vector data may be data containing vector values. The input vector data may be referred to as an embedded vector. When the sub-matrix according to some example embodiments is a part of a weight matrix of an AI model, the same input vector data may be input to each of the plurality of CXL processing devices, but example embodiments are not limited thereto. For example, the first input vector data 221 and the second input vector data 222 may be identical to each other. According to other example embodiments, when the sub-matrices for the plurality of CXL processing devices are identical to each other, the input vector data input to the plurality of CXL processing devices may be identical to or different from each other.

[0031] When the matrix multiplication calculation is performed in each of the plurality of CXL processing devices, output vector data may be generated by each of the plurality of CXL processing devices. The output vector data may be data containing vector values. For example, the first CXL processing device 110_1 may generate first output vector data 241, and the second CXL processing device 110_2 may generate second output vector data 242, and so on.

[0032] Each of the plurality of CXL processing devices may send at least one data packet and / or at least one interrupt signal to the CXL switch 100. According to some example embodiments, the data packet may include output vector data and characteristic data, but is not limited thereto. The characteristic data may include information desired and / or necessary for synchronization in the CXL switch 100. The information desired and / or necessary for synchronization may include, for example, the type of calculation, the length of the embedded vector (e.g., output vector data), the starting address of the embedded vector, information about each CXL processing unit (e.g., ID, etc.), model information, etc. For example, the first CXL processing device 110_1 may provide the first data packet PKT1 and the first interrupt signal IRT1, etc. to the CXL switch 100. The second CXL processing device 110_2 may provide the second data packet PKT2 and the second interrupt signal IRT2, etc. to the CXL switch 100.

[0033] The CXL switch 100 may receive one or more data packets and / or one or more interrupt signals from a plurality of CXL processing devices. The output vector data of the data packet may be stored in the memory 102. For example, the first vector data VD1 and the second vector data VD2 may be stored in the memory 102. The first vector data VD1 may correspond to the first output vector data 241, and the second vector data VD2 may correspond to the second output vector data 242, but example embodiments are not limited thereto. The CXL switch 100 may generate at least one instruction signal based on the characteristic data in the received data packet in response to the interrupt signal. The instruction signal may include, for example, an address of the memory 102, a length of an embedded vector, a starting address of an embedded vector, calculation information, model information, etc. For example, the control logic 101 may generate a first instruction signal INST1 based on the first characteristic data of the first data packet PKT1, and store the first instruction signal INST1 in the memory 102. For example, the control logic 101 may generate a second instruction signal based on the second characteristic data of the second data packet PKT2, and store the second instruction signal in the memory 102.

[0034] refer to Figure 2B , each of the plurality of CXL processing devices (e.g., the first CXL processing device 110_1 and the second CXL processing device 110_2, etc.) may stand by until a synchronization completion signal is received from the CXL switch 100. The CXL switch 100 may perform at least one calculation on the stored vector data based on the at least one instruction signal. For example, the control logic 101 may output a scheduling control signal SCNT to the memory 102. The scheduling control signal SCNT may be a signal instructing the memory 102 to output the first instruction signal INST1 stored in the memory 102. The memory 102 may output the first instruction signal INST1 in response to the scheduling control signal SCNT. In addition, the control logic 101 may control and / or instruct the memory 102 to output the first vector data VD1 and the second vector data VD2 stored in the memory 102. The calculation logic 103 may receive the first instruction signal INST1 from the memory 102, and the calculation logic 103 may obtain the first vector data VD1 and the second vector data VD2 stored in the memory 102. The first vector data VD1 may correspond to the first output vector data 241, and the second vector data VD2 may correspond to the second output vector data 242, but is not limited thereto. The calculation logic 103 may determine the calculation operation 250 by decoding the first instruction signal INST1. For example, the calculation operation 250 may be a sum operation. However, example embodiments of the inventive concept are not limited thereto, and the calculation operation 250 may include one or more of various types, such as an ADD operation, a MAX operation, etc. Figure 2B, it is assumed that the calculation operation 250 is a sum operation. The calculation logic 103 may generate synchronized vector data SVD by calculating (eg, summing) the first output vector data 241 and the second output vector data 242, but example embodiments are not limited thereto.

[0035] refer to Figure 2C , the calculation logic 103 may output and / or sequentially output the synchronization completion signal SCS and the synchronized vector data SVD, but is not limited thereto. For example, the calculation logic 103 may first output the synchronization completion signal SCS. The synchronization completion signal SCS may be sent to the first CXL processing device 110_1 and the second CXL processing device 110_2. The first CXL processing device 110_1 and the second CXL processing device 110_2 may confirm and / or determine that the vector data is synchronized in response to the synchronization completion signal SCS. Once the calculation operation 250 is completed, the synchronized vector data SVD may be sent (e.g., sent in parallel) to each of the plurality of CXL processing devices, but is not limited thereto. For example, the calculation logic 103 may store the synchronized vector data SVD in the memory 102. The control logic 101 may provide at least one memory control signal MCNT to the memory 102. The memory control signal MCNT may be a signal for controlling the memory 102 to output the synchronized vector data SVD, but is not limited thereto. The memory 102 may output the synchronized vector data SVD in response to the memory control signal MCNT. Each of the plurality of CXL processing devices may receive the synchronized vector data SVD and perform one or more additional calculations using the synchronized vector data SVD. According to some example embodiments, the first CXL processing device 110_1 and the second CXL processing device 110_2 may additionally perform calculations based on the synchronized vector data SVD. For example, the first CXL processing device 110_1 and the second CXL processing device 110_2 may generate a new embedded vector by calculating the average value of each row of the vector data, but is not limited thereto. The first CXL processing device 110_1 and the second CXL processing device 110_2 may send an index indicating a result value for a query provided by the CXL host 10 to the CXL host 10 through the CXL switch 100, but example embodiments are not limited thereto. Likewise, the first CXL processing device 110_1 and the second CXL processing device 110_2 may send data indicating a matrix result value, etc. to the CXL host 10 through the CXL switch 100.

[0036] Figure 3 is a block diagram illustrating control logic according to at least one example embodiment.

[0037] refer to Figure 3, the control logic 101 may run at least one interrupt routine in response to at least one interrupt signal. An interrupt routine may refer to a series of operations for encoding information expected and / or necessary for synchronization included in a data packet into an instruction signal and storing the encoded instruction signal in an instruction queue. For this purpose, the control logic 101 may include an interrupt processor 310, an encoder 320, a scheduler 330, and / or a controller 340, but example embodiments are not limited thereto. According to some example embodiments, one or more of the interrupt processor 310, the encoder 320, the scheduler 330, and / or the controller 340, etc. may be implemented as a processing circuit. The processing circuit may include: hardware or a hardware circuit including a logic circuit; a hardware / software combination, such as a processor running software and / or firmware; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc., but is not limited thereto.

[0038] The interruption handler 310 may output at least one call signal in response to at least one interruption signal. The call signal may be a signal for enabling the encoder 320, but is not limited thereto. The call signal may be sent to the encoder 320, etc.

[0039] The encoder 320 may encode at least one instruction signal from the characteristic data in response to the call signal (eg, encode the characteristic data or a portion thereof to generate at least one instruction signal). The encoder 320 may then send the at least one instruction signal to the memory 102 .

[0040] The scheduler 330 may monitor the memory 102 and perform at least one scheduling operation. The scheduling operation may be an operation for determining the order of outputting one or more instruction signals stored in the memory 102 according to and / or based on the characteristics of the CXL processing device, etc., and outputting the one or more instruction signals according to and / or based on the determined and / or expected order. The instruction signals stored in the memory 102 may be output from the memory 102 to the computing logic 103 through the at least one scheduling operation.

[0041] The controller 340 may control the memory 102. For example, the controller 340 may control the memory 102 to output the output vector data (e.g., the first vector data VD1 and the second vector data VD2, etc.) stored in the memory 102. For example, the controller 340 may control the memory 102 to provide synchronized vector data SVD to a plurality of CXL processing devices, but example embodiments are not limited thereto.

[0042] Figure 4is a block diagram illustrating a memory according to at least one example embodiment.

[0043] refer to Figure 4 , the memory 102 may include a first buffer 410 and a second buffer 420, but example embodiments are not limited thereto.

[0044] The first buffer 410 may temporarily store output vector data (eg, first vector data VD1 and second vector data VD2, etc.) and synchronized vector data SVD. The first buffer 410 may be referred to as a memory buffer.

[0045] The second buffer 420 may queue the instruction signal in sequence. According to some example embodiments, the second buffer 420 may be implemented as a queue (and / or instruction queue) including a plurality of entries. However, the example embodiments of the inventive concept are not limited thereto. According to some example embodiments, the scheduler 330 may monitor the instruction queue of the memory 102, etc.

[0046] Figure 5 is a block diagram illustrating computing logic according to at least one example embodiment.

[0047] refer to Figure 5 , the computing logic 103 may include a decoder 510 and first to mth computing blocks 520_1, 520_2, ..., and 520_m, but is not limited thereto. m may be an integer equal to or greater than 2. According to some example embodiments, one or more of the decoder 510 and the first to mth computing blocks, etc. may be implemented as a processing circuit. The processing circuit may include: hardware or a hardware circuit including a logic circuit; a hardware / software combination, such as a processor running software and / or firmware; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc., but is not limited thereto.

[0048] The decoder 510 may decode at least one instruction signal to determine at least one computing operation.

[0049] At least one of the first to mth calculation blocks 520_1, 520_2, ..., and 520_m may perform arithmetic operations according to and / or based on the decoded instruction signal. The first to mth calculation blocks 520_1, 520_2, ..., and 520_m may be implemented as hardware logic calculators for performing different calculation operations. At least one of the first to mth calculation blocks 520_1, 520_2, ..., and 520_m may send synchronized vector data SVD to the memory 102.

[0050] Figure 6 is a block diagram illustrating a CXL processing apparatus according to at least one example embodiment.

[0051] refer to Figure 6 According to some example embodiments, the CXL processing device 600 may be implemented as a CXL-Near Memory (PNM), but is not limited thereto. The CXL-PNM may be used, for example, to process data in an AI model such as an LLM model. The CXL processing device 600 may include a CXL controller 610, a PNM 611, an interface 612, and / or a plurality of device memories 620 and 630, etc., but is not limited thereto. According to some example embodiments, one or more of the CXL controller 610, the PNM 611, the interface 612, and / or the plurality of device memories 620 and 630, etc. may be implemented as a processing circuit. The processing circuit may include: hardware or a hardware circuit including a logic circuit; a hardware / software combination, such as a processor running software and / or firmware; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc., but is not limited thereto.

[0052] The CXL controller 610 (eg, a memory controller, a memory processing circuit, etc.) may communicate with the plurality of device memories 620 and 630 through an interface 612. The CXL controller 610 may control each of the plurality of device memories 620 and 630 through the interface 612.

[0053] The PNM 611 may perform data processing operations. The PNM 611 may perform mathematical operations such as matrix calculations and / or vector calculations, but is not limited thereto. According to some example embodiments, the PNM 611 may include at least one register storing information about sub-matrices desired and / or required for a desired mathematical operation such as a matrix multiplication calculation. The PNM 611 may send an interrupt signal and / or a data packet to the CXL switch 100. According to some example embodiments, the CXL controller 610 and the PNM 611 may be integrated into one semiconductor chip, but example embodiments of the inventive concept are not limited thereto.

[0054] The plurality of device memories 620 and 630 may be implemented as, for example, volatile memories, but are not limited thereto, and, for example, one or more of these device memories may be nonvolatile memory devices.

[0055] and Figure 6 Different from the CXL processing device shown in the figure, the CXL processing device according to other example embodiments may be implemented as a GPU based on CXL. Alternatively, the CXL processing device according to other example embodiments may also be implemented as an NPU based on FPGA design, etc.

[0056] Fig. 7A and Figure 7B is a diagram for describing a data flow according to a comparative example and at least one example embodiment. In detail, Fig. 7A is a diagram for describing a data flow according to a comparative example, and Figure 7B is a diagram for describing a data flow according to at least one example embodiment.

[0057] refer to Fig. 7A, a plurality of CXL-PNM devices 721a, 722a, 723a, and / or 724a, etc. may be configured as a memory pool by being connected to and / or connected below the CXL switch 710a, but example embodiments are not limited thereto, and for example, there may be a greater or lesser number of CXL-PNM devices, etc. Therefore, the CXL-PNM devices 721a, 722a, 723a, and / or 724a, etc. may process vector data of an AI model such as an LLM model, etc. in parallel, but are not limited thereto. The processing operation performed by each of the CXL-PNM devices 721a, 722a, 723a, and 724a may correspond to a portion of the overall processing operation of the AI ​​model. Therefore, it is desirable and / or necessary to synchronize the vector data processed by the CXL-PNM devices 721a, 722a, 723a, and / or 724a, etc. To this end, some CXL-PNM devices 722a, 723a, and 724a (e.g., a subset of CXL-PNM devices, etc.) from among a plurality of CXL-PNM devices 721a, 722a, 723a, and 724a (e.g., a set of CXL-PNM devices, etc.) may each include vector data processed in a data packet, respectively, and transmit each data packet to a specific CXL-PNM device (e.g., a desired and / or central CXL-PNM device, such as the CXL-PNM device 721a) through the CXL switch 710a. Then, the specific CXL-PNM device (e.g., the CXL-PNM device 721a) may synchronize the partially processed vector data (e.g., vector data partially processed by the subset of CXL-PNM devices, etc.) included in the received data packet, and retransmit the data packet containing the synchronized vector data to some CXL-PNM devices 722a, 723a, and 724a (e.g., a subset of CXL-PNM devices) through the CXL switch 710a. During the above process, at least one hop (e.g., network hop, etc.) occurs. A hop is a portion of a path between a source and a destination in a computer network (e.g., a segment of a network path). For example, a data packet is passed from a source to a destination through a bridge, a router, and / or a gateway, etc. (e.g., a network device and / or a network appliance, etc.), wherein a hop occurs each time the data packet moves to the next network device. In the case (and / or path) where a data packet is moved from a CXL-PNM device to the CXL switch 710a, one hop occurs. In addition, in the case where a data packet moves from the CXL switch 710a to a CXL-PNM device, one hop occurs, so when each data packet moves from some CXL-PNM devices 722a, 723a, and 724a (e.g., a subset of CXL-PNM devices) to the CXL switch 710a, three hops occur.In addition, when the respective data packets are moved from the CXL switch 710a to a specific CXL-PNM device (e.g., the CXL-PNM device 721a), three hops occur. In other words, when the data packets are sent from some CXL-PNM devices 722a, 723a, and 724a (e.g., a subset of the CXL-PNM devices) to a specific CXL-PNM device (e.g., the CXL-PNM device 721a) through the CXL switch 710a, six hops occur. In this regard, even when the data packets containing synchronized vector data are sent from a specific CXL-PNM device (e.g., the CXL-PNM device 721a) to some CXL-PNM devices 722a, 723a, and 724a (e.g., a subset of the CXL-PNM devices), six hops occur. According to. Fig. 7A In the packet transmission process, when all packets are sent to one CXL-PNM device (eg, CXL-PNM device 721a), a bottleneck may occur in a path including the CXL-PNM devices and CXL switch 710a.

[0058] refer to Figure 7B , the CXL-PNM devices 721b, 722b, 723b, and 724b are configured as a memory pool under the CXL switch 710b, so that vector data (e.g., embedding vectors) of the AI ​​model can be processed partially and / or in parallel. The CXL switch 710b according to at least one example embodiment of the inventive concept can perform Fig. 7A In another embodiment, the synchronization processing is performed by the CXL-PNM device 721a (e.g., a specific CXL-PNM device, a desired CXL-PNM device, a central CXL-PNM device, etc.) instead of the CXL-PNM devices 721b, 722b, 723b, and 724b (e.g., a subset of the CXL-PNM devices). In other words, the vector data processed by each of the CXL-PNM devices 721b, 722b, 723b, and 724b may be sent to the CXL switch 710b, the CXL switch 710b may generate synchronized vector data by performing the synchronization operation, and the CXL switch 710b may resend the synchronized vector data to each of the CXL-PNM devices 721b, 722b, 723b, and 724b. In this case, when the data packet is sent from the CXL-PNM devices 721b, 722b, 723b, and 724b to the CXL switch 710b, four hops occur. Likewise, when a packet containing synchronized vector data is sent from CXL switch 710b to CXL-PNM devices 721b, 722b, 723b, and 724b, four hops occur. Figure 7B The data packet transmission process is Fig. 7ACompared to the comparative example of FIG. 7 , relatively fewer hops may occur, thereby reducing the occurrence of bottlenecks. Likewise, congestion that may occur in the network path between the CXL-PNM device and the CXL switch 710 b may be reduced and / or prevented. Likewise, latency may be reduced in the case of processing a large amount of data. Likewise, data may be processed quickly and efficiently in calculations and / or complex calculations for processing a large amount of data.

[0059] Figure 8 is a flow chart of a method of operating a CXL switch according to at least one example embodiment.

[0060] refer to Figure 8 A method of operating a CXL switching device according to at least one example embodiment of the inventive concept is to operate Figure 1 The method of the CXL switch 100 may include operation S10, operation S20 and / or operation S30, etc., but is not limited thereto.

[0061] In operation S10, the CXL switch device receives a plurality of data packets (e.g., data packets, etc.) and at least one interrupt signal from a plurality of CXL processing devices. Each data packet may include vector data and characteristic data. According to some example embodiments, operation S10 may include an operation of receiving a first data packet and a first interrupt signal from a first CXL processing device and an operation of receiving a second data packet and a second interrupt signal from a second CXL processing device, but is not limited thereto. For example, referring to Figure 2A The CXL switch 100 receives a first data packet PKT1 and a first interrupt signal IRT1 from the first CXL processing device 110_1. Then, the CXL switch 100 receives a second data packet PKT2 and a second interrupt signal IRT2 from the second CXL processing device 110_2.

[0062] In operation S20, the CXL switch device synchronizes the vector data by performing calculations on the vector data based on the plurality of data packets and the interrupt signal. Figure 2A and Figure 2B The CXL switch 100 generates an instruction signal in response to an interrupt signal received from (one or more) CXL processing devices, and synchronizes the vector data by calculating the vector data of each data packet based on the generated instruction signal.

[0063] In operation S30, the CXL switching device outputs synchronized vector data to a plurality of CXL processing devices. Figure 2C, the calculation logic 103 stores the synchronized vector data SVD in the memory 102. The control logic 101 provides a memory control signal MCNT to the memory 102. The memory 102 outputs the synchronized vector data SVD in response to the memory control signal MCNT. The synchronized vector data SVD is transmitted to the first CXL processing device 110_1 and the second CXL processing device 110_2 in parallel, but is not limited thereto, and for example, the synchronized vector data may be transmitted serially.

[0064] According to some example embodiments of the inventive concept, the method of operating a CXL switching device may further include an operation of outputting synchronization completion signals to the plurality of CXL processing devices. According to some example embodiments, the operation of outputting synchronization completion signals to the plurality of CXL processing devices may be performed before operation S30, but is not limited thereto.

[0065] Fig. 9 Is used to describe Figure 8 0048 ] A flowchart of at least one example embodiment of operation S20 of FIG.

[0066] refer to Fig. 9 , operation S20 may include operation S210, operation S220 and / or operation S230, but is not limited thereto.

[0067] In operation S210, the CXL switch buffers vector data. Figure 2A and Figure 4 , first vector data VD1 corresponding to the first output vector data 241 is stored in the second buffer 420 , and second vector data VD2 corresponding to the second output vector data 242 is stored in the second buffer 420 , but example embodiments are not limited thereto.

[0068] In operation S220, the CXL switch generates at least one instruction signal based on the characteristic data in response to the interrupt signal. Figure 2A and Figure 2B , the control logic 101 generates a first instruction signal INST1 based on the first characteristic data of the first data packet PKT1, and the first instruction signal INST1 is stored in the memory 102, but is not limited thereto.

[0069] In operation S230, the CXL switch generates synchronized vector data by performing a calculation operation according to at least one instruction signal. Figure 2A and Figure 2B, the control logic 101 outputs a scheduling control signal SCNT to the memory 102. The memory 102 outputs a first instruction signal INST1 in response to the scheduling control signal SCNT. The control logic 101 controls the memory 102 to output the first vector data VD1 and the second vector data VD2 stored in the memory 102. The calculation logic 103 receives the first instruction signal INST1, the first vector data VD1 and the second vector data VD2 from the memory 102. The calculation logic 103 determines the calculation operation 250 by decoding the first instruction signal INST1. The calculation logic 103 generates synchronized vector data SVD by calculating (e.g., summing) the first output vector data 241 and the second output vector data 242, but is not limited thereto.

[0070] Fig.10 According to at least one example embodiment Fig. 9 Flow chart of operation S220.

[0071] refer to Fig.10 , operation S220 may include operation S221, operation S222, operation S223 and / or operation S224, but example embodiments are not limited thereto.

[0072] In operation S221, the CXL switch outputs at least one call signal in response to at least one interrupt signal. Operation S221 may be performed by Figure 3 The interrupt handler 310 executes, but is not limited to this.

[0073] In operation S222, the CXL switch encodes at least one instruction signal from the characteristic data in response to the at least one call signal. Operation S222 may be performed by Figure 3 The encoder 320 is executed, but is not limited to this.

[0074] In operation S223, the CXL switch queues at least one encoded command signal. Operation S223 may be performed by Figure 3 The encoder 320 is executed, but is not limited to this.

[0075] In operation S224, the CXL switch device outputs the queued instruction signal according to and / or based on the scheduling order. For example, the scheduling order may include scheduling information related to processing of one or more instruction signals and / or data packets containing vector data received from a plurality of CXL processing devices, but is not limited thereto. Operation S224 may be performed by Figure 3 The scheduler 330 executes, but is not limited to this.

[0076] Fig.11 is used to describe the invention according to at least one example embodiment Fig. 9 Flow chart of operation S230.

[0077] refer to Fig.11 Operation S230 includes operation S231 and operation S232, but is not limited thereto.

[0078] In operation S231, the CXL switch decodes at least one instruction signal to confirm and / or determine at least one computing operation. For example, the CXL switch may determine and / or confirm the type of computing operation to be performed based on the decoded at least one instruction signal. Operation S231 may be performed by Figure 5 The decoder 510 performs the above operations, but example embodiments are not limited thereto.

[0079] In operation S232, the CXL switch performs an operation according to and / or based on the decoded instruction signal. Operation S232 may be performed by Figure 5 At least one of the first to mth computing blocks 520_1, 520_2, ..., and 520_m is executed.

[0080] While various example embodiments of the inventive concepts have been particularly shown and described, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the appended claims.

Claims

1. A system based on computing a fast link, the system comprising: A plurality of computing fast link processing devices, each of which is configured to: performing a matrix multiplication calculation based on the input vector data and the sub-matrix; as well as outputting at least one interrupt signal and at least one data packet based on the result of the matrix multiplication calculation, each of the at least one data packet comprising output vector data and characteristic data associated with the output vector data; and A fast link switching device is configured to: synchronizing the output vector data to generate synchronized vector data, the synchronizing comprising performing a computational operation on the output vector data based on the at least one interrupt signal and the at least one data packet; as well as The synchronized vector data is provided to the plurality of compute fast link processing devices.

2. The system according to claim 1, wherein: The computing fast link switching device comprises: a memory configured to store the output vector data; and A processing circuit, the processing circuit being configured to: generating at least one instruction signal based on the characteristic data in response to the at least one interrupt signal, and storing the at least one instruction signal in the memory; performing said computing operation based on said at least one stored instruction signal; generating the synchronized vector data based on a result of the computing operation; and The synchronized vector data is stored in the memory.

3. The system according to claim 2, wherein: The processing circuit is further configured to: outputting at least one call signal in response to the at least one interrupt signal; responsive to the at least one call signal, encoding the at least one instruction signal from the characteristic data and sending the at least one instruction signal to the memory; performing a scheduling operation to output the at least one instruction signal stored from the memory to the processing circuit; as well as The synchronized vector data stored in the memory is provided to the plurality of compute fast link processing devices.

4. The system according to claim 2, wherein: The processing circuit is further configured to: decoding the at least one instruction signal to determine a type of the computing operation; performing the computing operation based on the decoded at least one instruction signal and the determined type of the computing operation; as well as The synchronized vector data is sent to the memory.

5. The system according to claim 2, wherein: The memory is further configured to: temporarily storing the output vector data and the synchronized vector data; and The at least one instruction signal is queued in sequence.

6. The system according to claim 1, wherein: The plurality of computing fast link processing devices include: a first computational fast link processing device configured to perform a first matrix multiplication calculation between a first sub-matrix of a weight matrix of an artificial intelligence model and first input vector data; and The second computing fast link processing device is configured to perform a second matrix multiplication calculation of the first input vector data and a second sub-matrix different from the first sub-matrix.

7. The system according to claim 1, wherein: The plurality of computing fast link processing devices include: a first computation fast link processing device configured to perform a first matrix multiplication calculation based on a first sub-matrix and first input vector data; and A second computation fast link processing device is configured to perform a second matrix multiplication calculation based on the first sub-matrix and second input vector data.

8. The system according to claim 1, wherein: Each of the plurality of computing fast link processing devices comprises: a plurality of device memories; and a memory processing circuit configured to control the plurality of device memories, and The matrix multiplication calculation is performed, and the at least one interrupt signal and the at least one data packet are sent to the computing fast link switching device.

9. A method of operating a computing fast link switching device, the method comprising: receiving a plurality of data packets and at least one interrupt signal from a plurality of compute fast link processing devices, wherein each of the plurality of data packets includes vector data and characteristic data associated with the vector data; synchronizing the vector data, the synchronizing comprising performing a computational operation on the vector data based on the plurality of data packets and the at least one interrupt signal; and The synchronized vector data is output to the plurality of compute fast link processing devices.

10. The method according to claim 9, wherein: Receiving the plurality of data packets and the at least one interrupt signal comprises: receiving a first data packet and a first interrupt signal from a first computing fast link processing device; and A second data packet and a second interrupt signal are received from a second computing express link processing device.

11. The method according to claim 9, wherein: Synchronizing the vector data includes: buffering the vector data; generating at least one instruction signal based on the characteristic data in response to the at least one interrupt signal; and The synchronized vector data is generated by performing the computational operation based on the at least one instruction signal.

12. The method according to claim 11, wherein: Generating the at least one instruction signal comprises: outputting at least one call signal in response to the at least one interrupt signal; encoding the at least one instruction signal from the characteristic data in response to the at least one call signal; queuing the at least one instruction signal; and The at least one queued instruction signal is output based on a scheduling order.

13. The method according to claim 11, wherein: Generating the synchronized vector data comprises: decoding the at least one instruction signal to determine a type of the computing operation; and The computing operation is performed based on the decoded at least one instruction signal and the determined type of the computing operation.

14. The method according to claim 9, further comprising: At least one synchronization completion signal is output to the plurality of computing fast link processing devices.

15. A network device, comprising: a memory configured to store vector data in a plurality of data packets received from a plurality of processing devices; and A processing circuit, the processing circuit being configured to: generating at least one instruction signal based on characteristic data in the plurality of data packets in response to a plurality of interrupt signals received from the plurality of processing devices, and storing the at least one instruction signal in the memory; determining a computing operation type based on the at least one instruction signal stored in the memory; synchronizing the vector data stored in the memory, the synchronizing comprising performing a computational operation on the vector data based on the determined computational operation type; as well as Output synchronized vector data.

16. The network device according to claim 15, wherein: The memory is further configured to: storing the vector data and the synchronized vector data; and The at least one instruction signal is queued.

17. The network device according to claim 15, wherein: The processing circuit is further configured to: outputting at least one call signal in response to the plurality of interrupt signals; encoding the at least one instruction signal from the characteristic data in response to the at least one call signal; performing a scheduling operation based on the characteristic data to output the at least one stored instruction signal; as well as The synchronized vector data is provided from the memory to the plurality of processing devices.

18. The network device according to claim 15, wherein: The processing circuit is further configured to: decoding the at least one instruction signal to determine an operation type of the computing operation; and The computing operation is performed based on the decoded at least one instruction signal and the determined operation type.

19. The network device according to claim 15, wherein: The processing circuit is further configured to: At least one synchronization completion signal and the synchronized vector data are sequentially outputted.

20. The network device according to claim 15, wherein: The network device also includes a computing fast link switch.

Images