PCIe switching chip realizing in-transmission calculation, switch and communication system with same

Abstract

This invention discloses a PCIe switching chip for in-transmission computation, and a switch and communication system incorporating the same. The switching chip includes a physical layer, a data link layer, a transaction layer, and a computation module. In response to the transaction layer parsing a specific prefix carrying a preset computation instruction in a data packet, the computation module parses the computation instruction to extract computation parameter fields. The computation module generates and executes a task to compute data in transmission based on these parameter fields, without waiting for all data to arrive before starting computation: it first performs computation on the currently received portion of the data to be computed to obtain intermediate computation results, and then aggregates the subsequently received data with the intermediate computation results. This invention achieves an innovative "computation while transmitting" mechanism by inserting a programmable computation path into the PCIe switch, reducing system latency and bandwidth consumption.

Description

Technical Field

[0001] This invention relates to the field of PCIe communication, belonging to the technical fields of high-speed interconnect interfaces, electronic information processing and heterogeneous computing architecture, and particularly to a PCIe switching chip for realizing computation during transmission and a switch and communication system having the same. Background Technology

[0002] With the rapid development of workloads such as large-scale artificial intelligence (AI) model training, high-performance computing (HPC), real-time big data analysis, and cloud computing platforms, the amount of data interaction between different computing and storage devices within the system is showing a high-speed growth trend, which makes the demand for communication bandwidth and intermediate computing constantly increasing. The traditional method of relying on CPU / GPU to perform all data preprocessing, aggregation, filtering and verification is no longer able to meet performance requirements.

[0003] Taking current mainstream heterogeneous computing systems as an example, devices such as CPUs, GPUs, FPGAs, SmartNICs, and high-performance NVMe SSDs are typically interconnected via the PCI Express (PCIe) bus. Due to its high bandwidth, low latency, and point-to-point topology, PCIe has become the most important interconnect standard between servers and accelerators. In typical data center servers, AI training nodes, or storage arrays, PCIe switches play a crucial role in "multi-device aggregation interconnection," connecting multiple GPUs, SSDs, or accelerator cards and forwarding Transaction Layer Packets (TLPs) between devices. As a core interconnect component for servers, AI training nodes, and high-performance storage systems, the main function of PCIe switches is to receive, parse, forward, and manage PCIe transactions. Taking the mainstream Broadcom PEX series and Microchip Switchtec series as examples, these switching chips all follow the basic PCIe protocol framework. Through layered collaboration between the Physical Layer (PHY), Data Link Layer (DLL), and Transaction Layer (TL), they achieve point-to-point communication between multiple downlink devices and the uplink Root Complex. In the traditional PCIe switch mechanism, when a device sends a TLP (Transaction Layer Packet), the data transmission is as follows... Figure 1 As shown.

[0004] However, as the number of GPUs continues to increase, the system bottleneck has shifted from "insufficient computing power" to "excessive data transport costs." Simultaneously, scenarios such as AI large-scale model training, distributed deep learning, multi-GPU All-Reduce, metadata processing in large-scale NVMe SSD storage systems, and micro-batch small operator computation within data centers are increasingly demanding low latency, intermediate data processing, and lightweight computational offload. In training clusters centered around GPU groups, the same gradient typically needs to be generated by multiple GPUs in parallel computation and summed and averaged before the next iteration, a process entirely dependent on communication efficiency. Similarly, storage systems, when performing data deduplication, verification, and filtering, also need to handle a large number of small data blocks moving between devices. In distributed GPU training, a gradient block typically requires: GPU to switch, to RC or GPU, back to switch, and finally back to GPU. These multiple round trips significantly increase PCIe traffic, raise overall latency, and involve CPU / GPU in meaningless data transport.

[0005] The disclosure of the above background technical content is only for the purpose of assisting in understanding the concept and technical solution of this application, and does not necessarily provide technical instruction. Summary of the Invention

[0006] The purpose of this invention is to provide a PCIe switching chip that enables computation during transmission. The PCIe switching chip integrates a programmable computing unit and performs calculations on data passing through the switching path, effectively reducing the number of data backhauls and rerouting on the PCIe link and improving bandwidth utilization.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A PCIe switching chip for implementing computation during transmission, comprising:

[0009] The physical layer, which is configured to receive data streams;

[0010] The data link layer is configured to receive the data stream sent by the physical layer and process it to obtain transaction layer data packets;

[0011] The transaction layer is configured to recognize transaction layer packets processed by the data link layer, including parsing specific prefixes reserved in the PCIe protocol;

[0012] The calculation module is configured to perform calculations on data in transit transaction layer data packets in the following manner:

[0013] In response to the transaction layer parsing that the specific prefix carries a preset calculation instruction, the calculation module parses the calculation instruction to extract the calculation parameter field therein;

[0014] The calculation module generates and executes a task to calculate the data in transmission based on the calculation parameter field, and does not need to wait for all the data to be processed before starting the calculation: first, it performs calculations on the currently received part of the data to be processed to obtain intermediate calculation results, and then aggregates the newly received data to be processed with the intermediate calculation results.

[0015] Furthermore, following any one or a combination of the aforementioned technical solutions, the computing module includes:

[0016] A parsing unit is configured to parse the calculation instructions carried by the specific prefix in order to extract the calculation parameter fields therein;

[0017] A computing controller is configured to determine or establish a computing session based on the corresponding transaction layer data packet according to the computing parameter field, and to generate computing tasks associated with the computing session according to the computing parameter field.

[0018] A data cache is configured to store the data payload in the transaction layer data packet, as well as intermediate and / or final computation results.

[0019] The computing core unit is configured to perform calculations on the data payload in the data cache according to the computing tasks generated by the computing controller, and obtain intermediate calculation results and / or final calculation results.

[0020] Furthermore, based on any or a combination of the aforementioned technical solutions, the data cache includes a payload buffer, an input buffer, and an aggregation buffer, wherein the payload buffer is configured to store the data payload in the transaction layer data packet, and the aggregation buffer is configured to store the intermediate calculation results obtained from the calculation;

[0021] The computing core unit performs calculations on the data payload in the payload buffer in a cyclic manner, including: reading the current payload data to be calculated from the payload buffer according to the computing task and storing it in the input buffer; performing the calculations set by the computing task on the data elements in the input buffer and the aggregation buffer, writing the new intermediate calculation results obtained from the calculations back to the aggregation buffer, and updating the aggregation state of the data elements in the computing session associated with the current computing task.

[0022] Furthermore, based on any or a combination of the aforementioned technical solutions, the computing controller generates computing tasks associated with the computing session, including the address of the input buffer, the address of the aggregation buffer, and the operation type;

[0023] If the PCIe switching chip manages multiple independent computing sessions at the same time, then the different computing sessions are associated with at least different aggregate buffer addresses.

[0024] Furthermore, following any one or a combination of the aforementioned technical solutions, determining whether the computing session has completed aggregation based on the aggregation state of the data elements in the computing session includes:

[0025] The calculation parameter fields parsed by the parsing unit include an aggregation identifier and a calculation descriptor; the parsing unit is also configured to parse the header information of the transaction layer data packets to determine the size of each transaction layer data packet.

[0026] The computing controller determines the target identifier within the aggregation range based on the computing descriptor, and assigns one or more transaction layer data packets with the same aggregation identifier to the same computing session, defining the computing task volume of the computing session as the sum of the sizes of the one or more transaction layer data packets;

[0027] The number of data elements that have completed the operation is counted, and the count is compared with the computational task of the corresponding computing session. If the count is less than the computational task, the computing session has not completed the aggregation. If the count reaches the computational task, the computing session has completed the aggregation, and the current intermediate operation result in the aggregation buffer is used as the final operation result.

[0028] Furthermore, following any one or a combination of the aforementioned technical solutions, the calculation parameter field parsed by the parsing unit also includes a result pattern, including:

[0029] If the result mode is the payload replacement mode, then after the calculation session completes the aggregation, the final calculation result replaces the data payload in the corresponding transaction layer data packet to obtain a new transaction layer data packet.

[0030] If the result mode is an append mode, then after the calculation session completes the aggregation, the final calculation result is appended to the corresponding transaction layer data packet to obtain a new transaction layer data packet;

[0031] If the result mode is parallel mode, then after the computation session completes the aggregation, another transaction layer data packet is constructed, with the final computation result as its data payload, and the corresponding data layer data packet and the newly constructed transaction layer data packet are used as a new data layer data packet.

[0032] Furthermore, following any one or a combination of the aforementioned technical solutions, after obtaining the new transaction layer data packet, the method further includes:

[0033] The data link layer and physical layer encapsulate the new transaction layer data packet to obtain a TLP data block conforming to the PCIe protocol format;

[0034] The TLP data block is sent out through the routed port, wherein the routed port information is determined by parsing the header information of the transaction layer data packet.

[0035] Furthermore, based on any or a combination of the aforementioned technical solutions, the calculation parameter field parsed by the parsing unit also includes an operation code identifier and a pattern representing the calculation type;

[0036] The operation code identifier matches the instruction set set in the expansion capability block of the configuration space corresponding to the PCIe switching chip;

[0037] The computing controller determines or establishes a computing session for the corresponding transaction layer data packet based on the aggregation identifier and operation code identifier.

[0038] Furthermore, following any or a combination of the aforementioned technical solutions, the parsing unit is further configured to parse the header information of the transaction layer data packet to determine the block identifier;

[0039] The computing controller determines or establishes a computing session for the corresponding transaction layer data packet based on the aggregation identifier, operation code identifier, and block identifier.

[0040] The PCIe switching chip manages one or more independent computing sessions at the same time.

[0041] Furthermore, following any one or a combination of the aforementioned technical solutions, if the task of calculation is to perform an averaging operation, the number of aggregations is recorded during / after each additive aggregation.

[0042] After obtaining the final aggregation result, a normalization calculation operation is performed to divide it by the number of aggregations to obtain the final calculation result of the averaging operation.

[0043] Furthermore, based on any or a combination of the aforementioned technical solutions, the computing core unit includes a RISC-V core and its firmware;

[0044] And / or, the parsing unit is a TLP Parser of a PCIe switch;

[0045] And / or, the specific prefix reserved in the PCIe protocol is the Vendor-Specific Prefix field.

[0046] Furthermore, following any one or a combination of the aforementioned technical solutions, if the transaction layer parses that the specific prefix does not carry a preset calculation instruction, then no calculation is performed on the data in transmission:

[0047] The transaction layer parses the header information of the transaction layer data packet to determine the port for routing selection;

[0048] The data link layer and physical layer repackage the transaction layer data packet to obtain a TLP data block conforming to the PCIe protocol format;

[0049] The TLP data block is sent out through the routed port.

[0050] According to another aspect of the present invention, a PCIe switch is provided, including a clock circuit, a power module, an EEPROM, a configuration space, and a PCIe switching chip as described above.

[0051] According to another aspect of the present invention, a communication system is provided, including an uplink device, a downlink device, and a PCIe switch as described above, wherein the uplink port of the switch is bidirectionally connected to the uplink device, and the downlink port of the switch is bidirectionally connected to the downlink device.

[0052] Furthermore, following any one or a combination of the aforementioned technical solutions, the uplink device writes a calculation instruction into its VSP field before sending the TLP data packet;

[0053] The PCIe switch's PCIe switching chip receives and parses the TLP data packet written with the calculation instruction to extract the calculation parameter field from the calculation instruction.

[0054] The PCIe switching chip generates and executes a task to calculate the TLP payload data in transmission based on the calculation parameter field, without waiting for all the calculation data to arrive before starting the calculation: first, it calculates the portion of the payload data that has been received to obtain intermediate calculation results, and then it aggregates the newly received payload data to be calculated with the intermediate calculation results.

[0055] The beneficial effects of the technical solution provided by this invention are as follows:

[0056] a. Integrating a programmable computing unit inside the PCIe switching chip and performing calculations on the data passing through the switching path reduces the intermediate computing load of the CPU / GPU, effectively reducing the number of data backhauls and reroutings on the PCIe link and improving bandwidth utilization.

[0057] b. It has a wide range of applications and can be applied to various computing scenarios that require high bandwidth and low latency data interaction, including multi-GPU collaborative artificial intelligence training clusters, high-speed interconnection between data center servers, accelerated interconnection of high-performance computing (HPC) nodes, resource pooling architecture of cloud computing platforms, data preprocessing and screening of large-scale storage systems (such as NVMe SSD arrays), and data exchange optimization between heterogeneous accelerators (GPUs, FPGAs, TPUs).

[0058] c. It can realize asynchronous aggregation of multi-source data, which is suitable for multi-GPU collaborative computing scenarios. The switching chip can manage multiple independent sessions internally and perform real-time accumulation and aggregation without waiting for all source data to arrive at the same time or before starting the calculation. It realizes the innovative mechanism of "computing while transmitting".

[0059] d. It does not require modification of the PCIe topology or changes to the host-side software stack. It can be independently integrated as a capability enhancement module for switching chips, which has high engineering feasibility. This allows switching chips to evolve from simple data forwarding devices into intelligent data processing nodes, thus having significant technical value and application prospects for AI / HPC / data center and other industry scenarios. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram illustrating the working mechanism of a traditional PCIe switch.

[0062] Figure 2 This is a schematic diagram of the workflow of a traditional PCIe switch.

[0063] Figure 3 A schematic diagram of the hardware structure of a PCIe switching chip provided for an exemplary embodiment of the present invention;

[0064] Figure 4 A schematic diagram illustrating the workflow of a PCIe switching chip implementing computation during transmission, provided as an exemplary embodiment of the present invention;

[0065] Figure 5 A schematic diagram of the TLP data structure provided for an exemplary embodiment of the present invention;

[0066] Figure 6A schematic diagram of the composition structure of a VSP field provided for an exemplary embodiment of the present invention;

[0067] Figure 7 A schematic diagram of a PCIe communication system provided as an exemplary embodiment of the present invention. Detailed Implementation

[0068] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0069] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0070] In traditional architectures, PCIe switches function solely as routing and forwarding devices, lacking any computational capabilities. When devices need to perform lightweight computations such as gradient aggregation, data filtering, feature extraction, verification, and encoding compression, even if these operations are not complex, data must be sent to the CPU or GPU for computation, and the results returned to the target device via PCIe. This model results in a large amount of data being transferred back and forth between multiple devices, consuming PCIe bandwidth, increasing overall system latency and energy consumption, and further burdening the CPU / GPU, preventing it from fully focusing on core computational tasks.

[0071] As can be seen from the above, although traditional PCIe switches play a crucial role in multi-device interconnection, their functionality has always been limited to transparent forwarding and protocol scheduling, without participating in any data content-related processing. Traditional PCIe switches lack computing power, cannot preprocess data or accelerate traditional switching chips, and are only responsible for forwarding TLPs. They do not parse the payload, perform any aggregation, filtering, compression, or other calculations as data passes through the switching chip, lack programmable data paths, and cannot support any on-the-fly computation. In AI, HPC, and storage scenarios, a large amount of lightweight computation (such as summation, averaging, filtering, and small operator processing) must be handled by the CPU / GPU, resulting in a waste of computing resources.

[0072] This invention aims to propose a solution for performing these lightweight calculations during data movement, so as to significantly reduce bandwidth consumption, reduce CPU / GPU execution load, and improve the performance and efficiency of the entire system.

[0073] RISC-V, as an open instruction set architecture (ISA), features customizability, scalability, and flexible implementation, making it particularly suitable for implementing lightweight, customizable computing units within a chip. This invention embeds one or more RISC-V computing cores into a PCIe switch, enabling preprocessing, vector accumulation, filtering, compression, or partial computation of data as it passes through the switch chip. This represents an architectural innovation aligned with industry trends. This "in-transit computing" or "in-switch computing" concept is similar to the in-network computing that has emerged in network switching chips in recent years. Its core idea is to complete some computations while data flows through the switching device, rather than dragging all data to the CPU / GPU for processing.

[0074] One of the technical problems that this invention aims to solve is how to use the Vendor-Specific Prefix (VSP) field of the PCIe protocol to carry computation instructions for each TLP without changing the PCIe protocol architecture, and perform lightweight computation on the payload data through the RISC-V computing unit inside the PCIe Switch, thereby reducing data movement, reducing system latency, improving bandwidth utilization, and forming a brand-new heterogeneous computing system.

[0075] This invention enables "computing while transmitting" of Transaction Layer Packet (TLP) payload data by embedding a RISC-V microprocessor unit within a PCIe switch. This significantly reduces the number of data transfers, lowers system latency, improves bandwidth utilization, and effectively offloads lightweight computing tasks from the host and accelerator. Therefore, it is suitable for building more efficient AI cluster interconnection systems, intelligent network devices, storage controllers, and future scalable distributed computing platforms.

[0076] In one embodiment of the present invention, a PCIe switching chip for implementing computation during transmission is provided, such as... Figure 3 As shown, the PCIe switching chip includes:

[0077] The physical layer is configured to receive data streams. It is responsible for electrical signal conversion and transmission. As the lowest layer, it directly processes electrical signals on the cable: at the receiving end, it converts analog signals on differential signal lines into digital bit streams, then descrambles and decodes them, and finally hands the recovered parallel data to the data link layer; at the transmitting end, it receives parallel data from the data link layer, scrambles and encodes it, and finally converts it into serial differential signals for transmission.

[0078] The data link layer is configured to receive the data stream sent by the physical layer and process it to obtain transaction layer data packets. The data link layer ensures reliable transmission of TLPs between adjacent devices: at the receiving end, it checks the Cyclic Redundancy Check (LCRC). If an error is found, it returns a negative acknowledgment (NACK) data link packet to the sender. Upon receiving the NACK, the sender retransmits the TLP from the replay buffer. If the check passes, it strips the sequence number and LCRC, submitting the "clean" TLP to the transaction layer. At the sending end, after receiving the TLP to be forwarded from the transaction layer, it generates a new sequence number, calculates a new LCRC, re-encapsulates it into a new TLPFrame, and then hands it over to the physical layer for transmission.

[0079] The transaction layer is configured to recognize transaction layer packets processed by the data link layer, including parsing specific prefixes reserved in the PCIe protocol. The transaction layer is the brain of the switching chip; it understands the meaning of TLPs and makes routing decisions, that is, it determines which destination port to send data to based on the address or ID in the TLP header. In addition, the transaction layer can also create and interpret the meaning of TLPs (read, write, configuration, etc.) for endpoint devices, and manage buffer credits to prevent receiver overload.

[0080] The calculation module is configured to perform calculations on data in transit transaction layer data packets in the following manner:

[0081] In response to the transaction layer parsing that the specific prefix carries a preset calculation instruction, the calculation module parses the calculation instruction to extract the calculation parameter field therein;

[0082] The calculation module generates and executes a task to calculate the data in transmission based on the calculation parameter field, and does not need to wait for all the data to be processed before starting the calculation: first, it performs calculations on the currently received part of the data to be processed to obtain intermediate calculation results, and then aggregates the newly received data to be processed with the intermediate calculation results.

[0083] One embodiment of the present invention is as follows: Figure 3 As shown, the computing module includes:

[0084] A parsing unit is configured to parse the calculation instructions carried by the specific prefix in order to extract the calculation parameter fields therein;

[0085] A computing controller is configured to determine or establish a computing session based on the corresponding transaction layer data packet according to the computing parameter field, and to generate computing tasks associated with the computing session according to the computing parameter field.

[0086] A data cache is configured to store the data payload in the transaction layer data packet, as well as intermediate and / or final computation results.

[0087] The computing core unit is configured to perform calculations on the data payload in the data cache according to the computing tasks generated by the computing controller, and obtain intermediate calculation results and / or final calculation results.

[0088] It should be noted that the division of units in the computing module is based solely on function and does not imply any limitation on hardware. For example, the parsing unit here can be an independent unit with parsing capabilities, or the TLPParser of the switch can be used as the parsing unit in this computing module.

[0089] Figure 2 This is a schematic diagram of the workflow of a traditional PCIe switch. Figure 4 and Figure 2 As can be seen from the comparison, the main difference between this embodiment and the traditional technology lies in the fact that if the parsing unit parses a specific prefix (VSP) of the TLP carrying computation instructions, it will change the traditional workflow. The following will explain... Figure 4 The following is a complete description of the workflow of the PCIe switching chip provided in this embodiment of the invention for implementing computation during transmission:

[0090] Step 1: The physical layer of the switching device receives the bit stream to be transmitted; the PCIe switching chip receives high-speed serial bit streams from upstream or downstream ports through its SerDes channel. The physical layer performs decoding, word alignment, signal recovery, and other processing on the bit stream according to the supported PCIe generation (e.g., PCIe 3.0 uses 8b / 10b encoding, and PCIe 4.0 / 5.0 uses 128b / 130b encoding), and switches the continuous data to the format specified by the data link layer.

[0091] Step 2: Data Link Layer Reassembles the TLP Frame; After the symbol sequence output from the physical layer enters the link layer, the switching chip reassembles and restores the order of the data according to the channel ID. The link layer is also responsible for: removing DLLPs (Data Link Layer Packets), checking and verifying the link CRC (LCRC), requesting retransmissions as needed, and ensuring that the final output is a correctly formatted and error-free TLP, ultimately obtaining a clean TLP that conforms to the specifications and can be processed by the transaction layer.

[0092] Step 3: The transaction layer parses the TLP header information. The transaction layer first identifies the TLP type and parses the header, including: transaction type (Memory Read / Write, Completion, Atomic, Config, etc.), address or routing information, Requester ID, Tag, attribute fields (Attr / TC), length and payload structure, whether it belongs to the uplink / downlink direction, and whether a completion or response packet is required. At this stage, the switching chip does not process the payload data but only relies on the header information to determine the target device or root complex of the transaction in the topology.

[0093] Step 4: Route decision, determine the target port; and parse the specific prefix reserved in the PCIe protocol; the core routing decision function of the switching chip is that the PCIe Switch determines which port the TLP should be forwarded to based on the address mapping range, Bus / Device / Function (BDF) path, internal routing table, BAR mapping of the downlink port and other information in the TLP Header. For example: memory access request → forwarded to the downlink port of the corresponding device, complete reply packet → returned to the original request, configuration access → forwarded uplink to the initiator of the TLP (such as GPU or RC).

[0094] The difference from traditional switch workflows is that the transaction layer also parses specific prefixes reserved in the PCIe protocol. Correspondingly, before the data processing begins, the vendor extension mechanism provided by the PCIe protocol—namely, the Vendor-Specific Extended Capability (VSEC) field in the configuration space and the Vendor-Specific Prefix field (hereinafter referred to as VSP) in the TLP—is used to complete the initial configuration of the switching chip's computing capabilities and the dynamic instruction issuance for each computing task, respectively. The core idea is to utilize the Vendor-Specific Prefix (VSP) reserved in the PCIe protocol and the Vendor-Specific Extended Capability (VSEC) field in the configuration space to implement dynamic instruction issuance and static computing capability configuration, respectively. VSEC (Vendor-Specific Extended Capability) is located in the PCIe device's configuration space. Its function is to be configured by the Root Complex (RC) or its driver during the system initialization phase, informing the switching chip of its internal pre-set computing capabilities and related parameters. By writing to the VSEC structure, the switch chip can be configured with its supported instruction set (opcode table), RISC-V computing unit function mapping, session and packet control registers, permission switches, operator numbers, and firmware entry addresses. The characteristic of VSEC is that it describes "what operations the switch chip can perform," representing a static, device-level capability setting that is not frequently modified during system operation due to changes in the TLP. The Vendor-Specific Prefix (VSP) is a 32-bit vendor-defined field extending the TLP prefix region, used to carry a dynamic computation task descriptor in each TLP. When data needs to be computed within the switch chip, the TLP initiator (e.g., GPU, accelerator card, or RC) writes a VSP field conforming to the definition in this embodiment into the TLP Prefix through its PCIe controller and related drivers when constructing the TLP. If the sender is a GPU, the GPU driver and its PCIe controller are responsible for generating and injecting the Prefix; if it is an RC, the RC-side PCIe controller inserts it.

[0095] Step 5: If the transaction layer parses a VSP containing a preset calculation instruction, it then parses the instruction to extract the calculation parameter fields. Figure 5The Vendor-Specific Prefix field shown is essentially a "computation descriptor" for the current TLP, indicating the specific computation task the switching chip should perform after parsing the TLP. The transaction layer can extract the computation parameter fields written by the sending end (such as GPU or RC) by parsing the VSP, such as... Figure 6 The following are the OPCODE_ID, MODE, RESULT_MODE, GROUP_ID, and LEN_IN_UNITS values:

[0096] OPCODE_ID represents the operation code identifier, which matches the instruction set set in the expansion capability block of the configuration space corresponding to the PCIe switching chip;

[0097] MODE represents the mode of the calculation type, such as summation, averaging, filtering, small operator processing, etc.

[0098] RESULT_MODE represents the result mode, which determines the form in which the final computation result is sent. This includes: if the result mode is payload replacement mode, then after the computation session completes aggregation, the final computation result replaces the data payload in the corresponding transaction layer data packet to obtain a new transaction layer data packet; if the result mode is append mode, then after the computation session completes aggregation, the final computation result is appended to the corresponding transaction layer data packet to obtain a new transaction layer data packet; if the result mode is parallel mode, then after the computation session completes aggregation, another transaction layer data packet is constructed, with the final computation result as its data payload, and the corresponding data layer data packet and the newly constructed transaction layer data packet are used as a new data layer data packet.

[0099] LEN_IN_UNITS represents the computation descriptor, which can include the length of the operation data, the registers or parameters required for the computation; GROUP_ID represents the aggregation identifier. By identifying the GROUP_ID corresponding to each TLP, the computation session to which it belongs is determined. Specifically, the computation controller determines the target identifier within the aggregation range based on LEN_IN_UNITS, and assigns one or more transaction layer data packets with the aggregation identifier matching the same aggregation range to the same computation session. The computation workload of this computation session is defined as the sum of the sizes of the one or more transaction layer data packets.

[0100] The computational parameter fields in the VSP above are written by the initiator of the TLP (such as GPU or RC) during the sending phase.

[0101] See Figure 4If the transaction layer parses out that the VSP does not contain any calculation instructions, it does not change the traditional behavior, that is, it does not perform calculations on the data in transit, and directly proceeds to forwarding according to the original path after the routing decision (see steps 9 and 10 below).

[0102] Step 6: Based on the extracted calculation parameter fields, the calculation controller determines or establishes a calculation session for the corresponding transaction layer data packet and generates a calculation task associated with the calculation session;

[0103] For example, the current computation task is to average the TLP payload data elements of GROUP_01 and GROUP_02. GROUP_01 is the aggregation identifier of the first device, and GROUP_02 is the aggregation identifier of the second device. The TLP data sent by the first device and the TLP data sent by the second device are successively sent to the same switching device. After going through steps 1 to 5 above, the computation controller can determine that the data with aggregation identifiers GROUP_01 and GROUP_02 belong to a computation session. If the corresponding session does not exist, the computation controller creates a new computation session and initializes the aggregation status of the computation session (not aggregated / aggregated). If the session already exists, the data block is added to the computation session as a new input. Subsequently, the TLP payload is stripped by the transaction layer and written to a dedicated data buffer via FIFO or internal DMA. This data buffer is used to store data fragments from different devices, as well as intermediate and / or final computation results of the current aggregation operation.

[0104] It should be noted that if the content of the computation descriptor is to perform filtering operations on the TLP payload data elements of GROUP_01 and GROUP_02, then another computation session needs to be created. That is to say, in addition to the aggregation identifier (GROUP_ID), the operation code identifier (OPCODE_ID) also jointly determines the computation session to which the data packet belongs.

[0105] In a further embodiment, the parsing unit is also configured to parse the header information of the transaction layer data packet to determine the block identifier; the compute controller, in addition to determining the aggregation identifier (GROUP_ID) and operation code identifier (OPCODE_ID), also determines or establishes a compute session for the corresponding transaction layer data packet based on the block identifier (CHUNK_ID); the PCIe switching chip manages one or more independent compute sessions simultaneously. During extended decoding, the switch's TLP Parser parses the VSP field to extract the compute parameter field written by the sender. Furthermore, it uses the Tag field (or a portion of the field in the address space) in the PCIe TLPHeader to generate an internally used block identifier CHUNK_ID to distinguish data blocks from different gradient fragments of the same GROUP_ID. This enables the switching chip to manage data streams submitted in parallel by multiple GPUs and accurately maintain the correspondence across multiple source sessions.

[0106] Step 7: The computing core unit performs calculations on the data payload in the data cache according to the computing task generated by the computing controller, obtains intermediate calculation results and / or final calculation results, and stores them in the data cache. In this embodiment, the computing core unit includes a RISC-V core and its firmware.

[0107] Specifically, the data cache includes a payload buffer, an input buffer, and an aggregation buffer. The stripped payload (payload data) is written to the payload buffer via FIFO or internal DMA. After data buffering is complete, the compute controller generates a compute task based on the parsed fields and session state, including the address of the input buffer, the address of the aggregation buffer, the operation type (such as vector accumulation, averaging, filtering, or compression), the operation length, and related mode parameters, and adds the task to the task queue of the RISC-V compute core unit. If the PCIe switching chip manages multiple independent compute sessions simultaneously, different compute sessions are associated with different aggregation buffer addresses, and different compute sessions can be associated with the same input buffer or different input buffers.

[0108] The aggregation buffer is configured to store intermediate computation results (the current intermediate computation result after the final aggregation is completed may become the final computation result).

[0109] The computational core unit performs operations on the data payload in the payload buffer in a cyclical manner. The RISC-V computational core unit operates in a "task-driven + element-level pipeline" manner: the computational unit retrieves tasks from the task queue, parses the instruction content (including OPCODE_ID, length LEN_IN_UNITS, mode MODE, etc.), and generates the corresponding addresses of the input buffer and the aggregation buffer; subsequently, under the management of the internal controller, the RISC-V core performs operations on the data in the input buffer and the aggregation buffer element by element, and writes the results back to the aggregation buffer, specifically including:

[0110] S701: Read the current load data to be calculated from the load buffer according to the calculation task, and store it in the input buffer; the length to be read is specified by the calculation instruction in VSP. In this embodiment, for example, if the specified operation data length is 8 bits, then the space of the input buffer is 8 bits. Before each calculation, the 8 bits of load data in the load buffer enter the input buffer and replace the original data in the input buffer.

[0111] S702: Perform the operations (such as vector accumulation, averaging, filtering, or compression) set by the computation task on the data elements in the input buffer and aggregation buffer.

[0112] For example, in this embodiment, taking the averaging task as an example, the addition operation is performed element by element on the data in the input buffer and the aggregation buffer; the ALU pipeline supports vector-level and scalar-level operations.

[0113] S703: Write the new intermediate calculation result obtained from the calculation back to the aggregation buffer, and update the aggregation state of the data elements in the computing session associated with the current computing task.

[0114] As can be seen, the cyclic execution of the task queue refers to: retrieving a task from the task queue → reading data from the input buffer → reading the current aggregation result (current intermediate calculation result) from the aggregation buffer → performing the specified vector operation (such as accumulation, finding the maximum / minimum, filtering, etc.) → writing back to the aggregation buffer → updating the session count and state → returning to the first step (retrieving a task from the task queue). Through this mechanism, the switching chip does not need to wait for data from multiple GPUs to arrive simultaneously, and can achieve asynchronous multi-source accumulation and averaging calculations by relying on the internal aggregation buffer and task queue mechanism. The arrival order of gradient blocks (CHUNK) from each GPU does not need to be strictly aligned, and the switch can aggregate data in real time according to the receiving order, thereby significantly reducing the round-trip and synchronization overhead of multi-source data.

[0115] S704: When the RISC-V core detects that all data from multiple source devices corresponding to a certain GROUP_ID has arrived, the RISC-V core completes the normalization calculation, that is, it automatically performs a normalization calculation by dividing by the number of sources: during / after each addition aggregation, the aggregation count is recorded; after obtaining the final aggregation result, a normalization calculation operation is performed by dividing it by the aggregation count to obtain the final calculation result for averaging. Finally, the RISC-V core writes the final calculation result back to the aggregation buffer and marks the task as completed.

[0116] Specifically, this can be confirmed by the following methods:

[0117] When parsing the header information of the TLP, the parsing unit can also determine the size of the data packets of each transaction layer;

[0118] The number of data elements that have completed the operation is counted, and the counted result is compared with the computational task of the corresponding computing session. If the counted result is less than the computational task, the computing session has not completed the aggregation. If the counted result reaches the computational task, the computing session has completed the aggregation, and the current intermediate operation result of the aggregation buffer is used as the final operation result, or, as in S704, the normalized operation result obtained by averaging is used as the final operation result.

[0119] Subsequently, the calculation unit returns a calculation completion flag to the calculation controller, providing a trigger signal for subsequent packetization and output processes. All computational logic for the averaging operation is consistent with the opcode mapping and computational capability table configured via VSEC during system initialization.

[0120] Step 8: After obtaining the final calculation result, obtain the new transaction layer data packet based on the RESULT_MODE field in the VSP.

[0121] Step 9: Repackaging. After the routing result is determined, the transaction layer of the switching chip will re-deliver the new transaction layer data packet to the link layer and physical layer for repackaging, including: regenerating LCRC (Cyclic Redundancy Check), adding necessary link layer management fields, adjusting channel allocation, and outputting the final TLP Frame conforming to the PCIe protocol format.

[0122] Step 10: Send to the target port. After repackaging, the PCIe switch sends the TLP to the target device or root complex in PCIe frame format according to the port selected by the routing (see the target port in step 4), completing a full switching process.

[0123] In one embodiment of the present invention, a PCIe switch is provided, including the PCIe switching chip as described above. The PCIe switching chip is the core of the switch. Besides this, the switch also includes several other important components, forming a complex system. These important components include a clock circuit, a power module, an EEPROM, a configuration space, etc. The clock circuit is typically a crystal oscillator, serving as a stable and accurate reference clock source for the switch. An internal or external phase-locked loop circuit is used to generate and synchronize the high-frequency clock signals required by each port. Since PCIe uses a source-synchronous clock, each link has its own clock, and these clocks must be precisely managed and synchronized.

[0124] The power module is responsible for supplying power to the entire switch chip and peripheral circuits: converting the input voltage (such as 12V) into various low-voltage power supplies (such as 3.3V, 1.8V, 1.0V, etc.) required by the switch chip core, I / O, etc.; in addition, the power module also uses capacitors and inductors to filter out power supply noise, ensuring clean and stable power supply, which is crucial for high-speed signals.

[0125] EEPROM is a non-volatile memory used to store the switch's firmware, configuration information, vendor ID, device ID, serial number, etc. The switch can load its initial configuration from it during system startup.

[0126] Each port of the switch has its own PCIe configuration space, making the switch itself appear as a multi-functional PCIe device in the system.

[0127] In one embodiment of the present invention, a communication system is provided, including an uplink device, a downlink device, and a PCIe switch as described above. The uplink port of the switch is bidirectionally connected to the uplink device, and the downlink port of the switch is bidirectionally connected to the downlink device. Figure 7 As shown, the uplink device can be a Root Complex (RC) or a GPU or accelerator card; the downlink device can be an endpoint device, and a switch can connect to multiple endpoint devices simultaneously.

[0128] The communication system provided in this embodiment belongs to the same inventive concept as the PCIe switching chip mentioned above, that is, before the uplink device sends TLP data packets, it writes calculation instructions into its VSP field.

[0129] The PCIe switch's PCIe switching chip receives and parses the TLP data packet written with the calculation instruction to extract the calculation parameter field from the calculation instruction.

[0130] The PCIe switching chip generates and executes a task to calculate the TLP payload data in transmission based on the calculation parameter field, without waiting for all the calculation data to arrive before starting the calculation: first, it calculates the portion of the payload data that has been received to obtain intermediate calculation results, and then it aggregates the newly received payload data to be calculated with the intermediate calculation results.

[0131] Upon detecting a computational need, the TLP data payload is stripped from the transaction layer and written to the internal data buffer via FIFO. Subsequently, the internal computation controller of the switch triggers the scheduling process of the RISC-V computation core unit based on the VSP field, including loading the corresponding instructions, reading input data, executing the specified operation (such as vector accumulation, averaging, filtering or compression), and writing the operation result back to the aggregation buffer or temporary storage area.

[0132] After the calculation is completed, the calculation controller selects the output method according to the RESULT_MODE field in the VSP: in replacement mode, the new calculation result directly replaces the original data payload; in append mode, the result is appended to the data payload; in parallel return mode, a new Completion TLP is constructed, and the calculation result is returned to the RC or a designated device. Subsequently, the switch enters the standard packet encapsulation stage (step 9), regenerates link layer fields such as LCRC / ECRC, and forwards the calculated new TLP to the target port according to the predetermined routing decision (step 10). The entire content of the PCIe switching chip embodiment is incorporated into this communication system embodiment by reference and will not be repeated here.

[0133] Through the above mechanism, this solution enables PCIe switches to perform simultaneous transmission and computation while forwarding data, without changing the PCIe protocol framework. This significantly reduces intermediate data handling and the light computational load on the CPU / GPU, thereby improving overall system performance and bandwidth utilization.

[0134] This invention aims to propose an architecture that integrates programmable computing capabilities within a PCIe switch and utilizes Vendor-Specific Prefix (VSP) and Vendor-Specific Extended Capability (VSEC) to jointly achieve dynamic command transmission and static capability configuration. Compared with the inherent limitations of traditional PCIe switches that lack data processing capabilities, the technical solution of this application has significant advantages in terms of system architecture, performance, and data link efficiency.

[0135] Firstly, this invention significantly reduces the overhead of data round-tripping within the system. In traditional architectures, cross-device data aggregation, filtering, statistics, or lightweight computations are all performed by the CPU or GPU, resulting in multiple round-trip transmissions of data on the PCIe link, causing bandwidth waste and communication latency. This invention moves these lightweight computations forward to the PCIe switch, allowing data to be processed on the forwarding path, thereby effectively reducing the number of data backhauls and rerouting. In multi-GPU distributed training or storage systems, this mechanism can eliminate a large number of repetitive read-back and write-back requirements, significantly improving the overall system bandwidth utilization.

[0136] Secondly, this invention reduces the intermediate computational load on the CPU / GPU. By integrating a RISC-V microprocessor within the switching chip, programmable vector-level operations, filtering, accumulation, or lightweight compression are achieved, freeing the CPU / GPU from handling numerous small operators or preprocessing tasks. This mechanism effectively frees up CPU / GPU resources, allowing them to focus more on core computational tasks and improving system parallelism and computational efficiency. In large-scale AI training scenarios, it can significantly improve gradient synchronization efficiency and reduce GPU idle waiting time.

[0137] Thirdly, this invention possesses high scalability and programmability. By configuring computing capabilities through VSEC and dynamically writing computing instructions through VSP, the switching chip can flexibly adjust its data processing logic according to different business scenarios without modifying the PCIe protocol structure. The dynamism of VSP allows different devices to customize the computing behavior of TLP as needed, while VSEC ensures the secure and standardized use of the internal computing resources of the switching chip. This combination of "static capabilities + dynamic tasks" gives the system high flexibility when expanding new operators and adapting to new applications.

[0138] Fourthly, this invention enables asynchronous aggregation of multi-source data, making it suitable for multi-GPU collaborative computing scenarios. Through mechanisms such as GROUP_ID and CHUNK_ID, the switching chip can internally manage multiple independent sessions and utilize the RISC-V core for real-time accumulation and aggregation, without waiting for all source data to arrive simultaneously. This asynchronous aggregation capability significantly reduces communication synchronization costs, enabling distributed training to maintain good scalability even on a large scale of nodes.

[0139] Fifthly, this invention fully complies with the PCIe specification, utilizes the Vendor-Specific field to achieve functional extensions without disrupting the existing protocol architecture, and is compatible with current mainstream PCIe switches, root complexes, GPU / NVMe / FPGA devices. Its deployment requires no modification to the PCIe topology or the host-side software stack, and it can be independently integrated as a switch chip capability enhancement module, demonstrating high engineering feasibility.

[0140] In summary, this invention achieves an innovative "computing while transmitting" mechanism by inserting programmable computing paths into PCIe switches. This reduces system latency, bandwidth consumption, and computing power utilization, and transforms the switching chip from a simple data forwarding device into an intelligent data processing node. Therefore, it has significant technical value and application prospects for industry scenarios such as AI / HPC / data centers.

[0141] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A PCIe switching chip for implementing computation during transmission, characterized in that, include: The physical layer, which is configured to receive data streams; The data link layer is configured to receive the data stream sent by the physical layer and process it to obtain transaction layer data packets; The transaction layer is configured to identify transaction layer packets processed by the data link layer, including parsing a specific prefix reserved in the PCIe protocol, wherein the specific prefix reserved in the PCIe protocol is the Vendor-Specific Prefix field; The calculation module is configured to perform calculations on data in transit transaction layer data packets in the following manner: In response to the transaction layer parsing that the specific prefix carries a preset calculation instruction, the calculation module parses the calculation instruction to extract the calculation parameter field therein; The calculation module generates and executes a task to calculate the data in transmission based on the calculation parameter field, and does not need to wait for all the data to be processed before starting the calculation: first, it performs calculations on the currently received part of the data to be processed to obtain intermediate calculation results, and then aggregates the newly received data to be processed with the intermediate calculation results. The computing module includes: The parsing unit, which is the TLP Parser of the PCIe switch, is configured to parse the calculation instructions carried by the specific prefix in order to extract the calculation parameter fields therein; A computing controller is configured to determine or establish a computing session based on the corresponding transaction layer data packet according to the computing parameter field, and to generate computing tasks associated with the computing session according to the computing parameter field. A data cache is configured to store the data payload in the transaction layer data packet, as well as the intermediate and / or final calculation results obtained from the calculation; the data cache includes a payload buffer, an input buffer, and an aggregation buffer, wherein the payload buffer is configured to store the data payload in the transaction layer data packet, and the aggregation buffer is configured to store the intermediate calculation results obtained from the calculation; The computing core unit includes a RISC-V core and its firmware. The computing core unit is configured to perform calculations on the data payload in the data cache according to the computing task generated by the computing controller, obtaining intermediate calculation results and / or final calculation results. The computing core unit performs calculations on the data payload in the payload buffer in a cyclic manner, including: reading the current payload data to be calculated from the payload buffer according to the computing task and storing it in the input buffer; performing the calculations set by the computing task on the data elements in the input buffer and the aggregation buffer, writing the new intermediate calculation results obtained back to the aggregation buffer, and updating the aggregation state of the data elements in the computing session associated with the current computing task.

2. The PCIe switching chip for in-transmission computation according to claim 1, characterized in that, The computing controller generates computing tasks associated with the computing session, including the address of the input buffer, the address of the aggregation buffer, and the operation type; If the PCIe switching chip manages multiple independent computing sessions at the same time, then the different computing sessions are associated with at least different aggregate buffer addresses.

3. The PCIe switching chip for in-transmission computation according to claim 1, characterized in that, Determining whether the computing session has completed aggregation based on the aggregation status of data elements in the computing session includes: The calculation parameter fields parsed by the parsing unit include an aggregation identifier and a calculation descriptor; the parsing unit is also configured to parse the header information of the transaction layer data packets to determine the size of each transaction layer data packet. The computing controller determines the target identifier within the aggregation range based on the computing descriptor, and assigns one or more transaction layer data packets with the same aggregation identifier to the same computing session, defining the computing task volume of the computing session as the sum of the sizes of the one or more transaction layer data packets; The number of data elements that have completed the operation is counted, and the count is compared with the computational task of the corresponding computing session. If the count is less than the computational task, the computing session has not completed the aggregation. If the count reaches the computational task, the computing session has completed the aggregation, and the current intermediate operation result in the aggregation buffer is used as the final operation result.

4. The PCIe switching chip for in-transmission computation according to claim 3, characterized in that, The calculation parameter fields parsed by the parsing unit also include result patterns, including: If the result mode is the payload replacement mode, then after the calculation session completes the aggregation, the final calculation result replaces the data payload in the corresponding transaction layer data packet to obtain a new transaction layer data packet. If the result mode is an append mode, then after the calculation session completes the aggregation, the final calculation result is appended to the corresponding transaction layer data packet to obtain a new transaction layer data packet; If the result mode is parallel mode, then after the computation session completes the aggregation, another transaction layer data packet is constructed, with the final computation result as its data payload, and the corresponding data layer data packet and the newly constructed transaction layer data packet are used as a new data layer data packet.

5. The PCIe switching chip for in-transmission computation according to claim 4, characterized in that, After receiving the new transaction layer data packet, it also includes: The data link layer and physical layer encapsulate the new transaction layer data packet to obtain a TLP data block conforming to the PCIe protocol format; The TLP data block is sent out through the routed port, wherein the routed port information is determined by parsing the header information of the transaction layer data packet.

6. The PCIe switching chip for in-transmission computation according to claim 3, characterized in that, The calculation parameter fields obtained by the parsing unit also include operation code identifiers and patterns representing the calculation type; The operation code identifier matches the instruction set set in the expansion capability block of the configuration space corresponding to the PCIe switching chip; The computing controller determines or establishes a computing session for the corresponding transaction layer data packet based on the aggregation identifier and operation code identifier.

7. The PCIe switching chip for in-transmission computation according to claim 6, characterized in that, The parsing unit is also configured to parse the header information of the transaction layer data packet to determine the block identifier; The computing controller determines or establishes a computing session for the corresponding transaction layer data packet based on the aggregation identifier, operation code identifier, and block identifier. The PCIe switching chip manages one or more independent computing sessions at the same time.

8. The PCIe switching chip for implementing in-transmission computation according to any one of claims 1 to 7, characterized in that, If the task is to perform an average calculation, then record the number of aggregations during / after each addition aggregation. After obtaining the final aggregation result, a normalization operation is performed by dividing it by the number of aggregations to obtain the final result of the averaging operation.

9. The PCIe switching chip for implementing in-transmission computation according to any one of claims 1 to 7, characterized in that, If the transaction layer parses the specific prefix and finds that it does not carry a preset calculation instruction, then no calculation is performed on the data in transit. The transaction layer parses the header information of the transaction layer data packet to determine the port for routing selection; The data link layer and physical layer repackage the transaction layer data packet to obtain a TLP data block conforming to the PCIe protocol format; The TLP data block is sent out through the routed port.

10. A PCIe switch, characterized in that, It includes a clock circuit, a power module, an EEPROM, a configuration space, and a PCIe switching chip as described in any one of claims 1 to 9.

11. A communication system, characterized in that, It includes an uplink device, a downlink device, and a PCIe switch as described in claim 10, wherein the uplink port of the switch is bidirectionally connected to the uplink device, and the downlink port of the switch is bidirectionally connected to the downlink device.

12. The communication system according to claim 11, characterized in that, Before sending TLP data packets, the uplink device writes a calculation instruction into its VSP field. The PCIe switch's PCIe switching chip receives and parses the TLP data packet written with the calculation instruction to extract the calculation parameter field from the calculation instruction. The PCIe switching chip generates and executes a task to calculate the TLP payload data in transmission based on the calculation parameter field, without waiting for all the calculation data to arrive before starting the calculation: first, it calculates the portion of the payload data that has been received to obtain intermediate calculation results, and then it aggregates the newly received payload data to be calculated with the intermediate calculation results.

Images