Multi-device data interaction system and server

Abstract

The application discloses a kind of multi-device data interaction system and server, it is related to computer technology field.Therein, system includes the first device with the persistent data storage function, the second device with data operation function, the third device with online storage function;Controller is connected with each functional device respectively, is configured to not participate in the data transmission between each functional device, only to each functional device is carried out operation management, and control establishes the data transmission channel between each functional device;Interconnector is connected with each functional device, controller respectively, under the control of controller, it provides the physical link of direct data transmission for each functional device, each functional device carries out data interaction without passing through central processing unit transfer case.The application can solve the problem that related technology multi-device interaction needs to pass through central processing unit transfer scheduling, realize the bidirectional direct connection of multi-device, without central processing unit transfer scheduling.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a multi-device data interaction system and server. Background Technology

[0002] With the rapid development of data-intensive applications such as high-performance computing and artificial intelligence, frequent data interactions occur between different types of devices within a computer system. To address the inefficiency of accelerating computing devices like GPUs relying on the CPU to transfer data from NVMe disks to GPU memory during large-scale or high-performance computing tasks, a PCIe switch is deployed on the GPU device. This switch connects the GPU, the storage backplane, and a PCIe bridge board, which in turn connects to both the CPU board and the storage backplane. This allows the GPU to directly access the NVMe disk without going through the CPU.

[0003] However, this method not only relies on the CPU for enabling controls such as device initialization and protocol parsing and conversion of storage read / write commands, but also, for tasks requiring large-scale data computation, GPU memory is limited, necessitating the use of system memory to temporarily cache intermediate data. Since system memory is a system-level resource, GPU access to system memory must go through the CPU. This method, requiring central processing unit intervention or scheduling, limits further improvements in data processing efficiency and cannot meet the ever-increasing demands for high-throughput and low-latency data transfer. Summary of the Invention

[0004] This invention provides a multi-device data interaction system and server. This system is a multi-device direct connection system that bypasses the central processing unit. It realizes fast, efficient and secure data transmission and operation control between multiple types of functional devices without the need for central processing unit relay scheduling, effectively improving data processing efficiency and meeting the growing demand for high throughput and low latency data transmission.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a multi-device data interaction system, comprising: Multifunctional devices include at least a first device with persistent data storage function, a second device with data processing function, and a third device with online storage function; The controller is connected to each functional device and is configured to: not participate in data transmission between functional devices, but only manage the operation of each functional device and control the establishment of data transmission channels between functional devices; The interconnect has multiple device ports, control ports, and physical links corresponding to each device port and control port. The first device, the second device, and the third device are connected to the respective device ports, and the controller is connected to the control port. Under the control of the controller, the interconnect provides physical links for direct data transmission to each functional device, enabling data interaction between the functional devices without the intervention of the central processing unit. During the execution of the target task, the second device, based on the direct memory access function provided by the interconnect, reads the data to be processed from the first device into its local computing space through the internal physical link of the interconnect. During the processing of the data to be processed, the intermediate processing data is directly written to the third device through the internal physical link of the interconnect. After completing the processing of the data to be processed, the processing result is written back to the first device through the internal physical link.

[0006] Another aspect of the present invention provides a server, including a central processing unit and the aforementioned multi-device data interaction system.

[0007] The advantages of the technical solution provided by this invention are that the controller does not participate in data transmission between functional devices, but only manages the operation of each functional device and controls the establishment of channels. Simultaneously, the interconnector provides a physical link for direct data transmission between the functional devices. Data transfer operations during task execution are all completed directly through the physical link provided by the interconnector. Data does not need to repeatedly enter and exit the main memory space connected to the central processing unit (CPU), nor does the CPU need to perform read / write processing. This allows for direct data transmission channels to be established between the first device with persistent data storage capabilities, the second device with data processing capabilities, and the third device with online storage capabilities, without the need for CPU intervention. This reduces data transfer links and memory read / write operations, lowers data transmission latency, and improves data transmission bandwidth utilization. Furthermore, device initialization, protocol parsing and conversion of storage read / write commands, and other enabling controls are completed by the controller, again without CPU intervention. Since the central processing unit (CPU) no longer intervenes in the data transfer process, its computing resources and the access bandwidth of the connected main memory are released, no longer becoming a factor limiting the data throughput performance between functional devices. This improves the data interaction rate between functional devices, thereby enhancing their data throughput performance and effectively improving user task execution efficiency, meeting the ever-increasing demand for high-throughput and low-latency data transmission. Furthermore, this invention also provides a corresponding server for multi-device data interaction systems, further enhancing the system's practicality. This server offers corresponding advantages. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the present invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of the structure of the multi-device data interaction system provided by the present invention; Figure 2 A schematic diagram of another exemplary structure of the multi-device data interaction system provided by the present invention; Figure 3 This is a schematic diagram of the server structure provided by the present invention; Figure 4 This is a schematic diagram of the multi-device data interaction process within the server of the present invention. Detailed Implementation

[0010] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. In this specification and the aforementioned drawings, the terms "first," "second," "third," "fourth," etc., are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0011] In scenarios such as artificial intelligence model training, big data processing, and high-performance computing, computer systems require various types of devices to work together to process the same task. During task processing, these devices need to perform a large amount of data reading, writing, and exchange. For example, when a server executes a user task, computing devices used to accelerate computation, storage devices providing persistent data storage, and storage devices used for temporary data caching need to frequently exchange data. Taking the data interaction between NVMe (Non-Volatile Memory Express) devices, GPUs (Graphics Processing Units), and memory as an example, NVMe devices read data into a buffer area of ​​memory through direct memory access. The CPU reads data from this buffer area and loads it into its internal cache or registers for processing. After processing, the data is written back to another buffer area of ​​memory. The GPU then moves the data from this buffer area into its video memory through direct memory access. The CPU's involvement in data transmission and processing generates significant computational overhead, consuming its own resources and leading to a decrease in overall operating efficiency. Data needs to pass through memory multiple times during transmission, making memory bandwidth a limiting factor. In large-scale model training and data read / write scenarios, insufficient bandwidth can severely impact the computing power of graphics processing units (GPUs) and the storage efficiency of hard drives. Furthermore, multiple intermediary stages result in high data transmission latency, making it difficult to meet the real-time requirements of high-performance computing.

[0012] Related technologies address these issues by establishing direct communication between different devices, but they still cannot achieve bidirectional direct connections between multiple devices, failing to meet the ever-increasing data transmission demands. For example, methods using PCIe (Peripheral Component Interconnect Express) P2P (Peer-to-Peer) for direct communication between GPU devices have limited device compatibility and scalability due to the PCIe topology. Another related technology addresses RDMA (Remote Direct Memory Access) communication in multi-plane networks by employing a hybrid NVLink and RDMA protocol to build a direct GPU connection channel, bypassing CPU data forwarding. Through memory address mapping and remote memory registration, it enables direct cross-plane GPU memory access, supporting efficient transmission of large amounts of data. This method targets scenarios involving cross-node communication in multi-GPU clusters, establishing direct connections between GPUs and combining remote direct memory access to achieve direct data access across the network, eliminating the central processing unit's role as an intermediary in GPU communication. However, this method is suitable for solving communication problems between multiple graphics processors, but it does not involve direct data interaction between storage devices and system main memory. Methods that implement a direct data path between the GPU and storage devices through GDS (Graphics Processing Unit Direct Storage) only achieve interaction between the GPU and storage, not high-speed interaction across the entire memory, NVMe, and GPU chain. For example, one related technology uses a PCIe switch to build a physical connection path between the GPU and NVMe, supporting GDS technology, allowing the GPU memory to directly access NVMe, reducing CPU intervention. This server architecture compatible with GPU direct storage allows graphics processing devices to directly access data in non-volatile storage devices, eliminating the central processing unit's involvement in data transmission. However, in this solution, the central processing unit is still responsible for device initialization, address allocation, and storage protocol processing, and is not completely free from dependence on the central processing unit. Another related technology enables direct data transfer between the GPU and FPGA (Field Programmable Gate Array) via an external bus interface, allowing the GPU to communicate directly with the FPGA. This solves the inefficiency problem of the current method, eliminating the need to temporarily store data in the main memory connected to the central processing unit, and completing data exchange using virtual pointers and direct memory access.Using an external bus interface (such as PCIe) to implement a direct data transfer system allows the GPU to communicate directly with the FPGA without storing data in CPU memory, achieving efficient data exchange using virtual pointers and DMA transfers. However, while this method enables direct connections between the GPU, FPGA, and peripherals, it does not establish a direct access path between NVMe and memory. The FPGA only acts as an auxiliary forwarding unit, and system-level management and protocol stack processing still rely on the CPU.

[0013] As can be seen from the above, in multi-device interaction scenarios, the data transmission and / or operation control processes of related technologies still rely on the CPU, making it impossible to achieve fast, efficient, and secure data transmission and control without CPU intervention. Therefore, this invention provides a multi-device direct-connection system that bypasses the central processing unit (CPU). The controller does not participate in data transmission between a first device with persistent data storage capabilities, a second device with data processing capabilities, and a third device with online storage capabilities; it is only responsible for operation management and channel establishment control. Simultaneously, the interconnector, under the control of the controller, provides a physical link for direct data transmission between the first, second, and third devices. This ensures that data interaction between the first, second, and third devices does not pass through the CPU, achieving fast, efficient, and secure data transmission and operation control between multiple devices without CPU-based scheduling, effectively improving data processing efficiency. After introducing the technical solution of this invention, various non-limiting embodiments of the invention will be described in detail below with reference to the accompanying drawings and specific implementation details.

[0014] Please see first. Figure 1 The present invention provides a multi-device data interaction system that can be deployed and run in a computer device that simultaneously includes a first device with persistent data storage function, a second device with data processing function, and a third device with online storage function, such as a user terminal device, a data center server, or an edge server. This embodiment includes the following: The multi-device data interaction system includes a device group 1, a controller 2, and an interconnector 3. The controller 2 is connected to each functional device in the device group 1, and the interconnector 3 is connected to each functional device in the device group 1 and the controller 2. The connection between the device group 1, the controller 2, and the interconnector 3 can be, for example, an electrical connection. The controller 2 is configured not to participate in data transmission between the functional devices, but only to manage the operation of each functional device and control the establishment of data transmission channels between them. Under the control of the controller 2, the interconnector 3 provides a physical link for direct data transmission between the functional devices, enabling data interaction between them without the intervention of a central processing unit.

[0015] Device group 1 includes multiple types of functional devices, which are collections of hardware entities in computer devices such as servers that perform different data processing tasks. The first device has persistent data storage capabilities; it is a non-volatile storage device with persistent data storage capabilities, such as a non-volatile memory high-speed interface standard solid-state drive (SSD), used to store data for a long time without data loss after power failure. The second device has data processing capabilities; it is a computing device with data processing capabilities, such as a graphics processor, tensor processing card, or field-programmable logic acceleration card, used to perform parallel computing tasks. The third device has online storage capabilities; it is a volatile storage device used to temporarily store data and instructions during system operation, such as DDR (Double Data Rate) memory, SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory), or Double Data Rate Synchronous Dynamic Random Access Memory, used to store computational data online during system operation; the data is lost after power failure.

[0016] In this embodiment, controller 2 refers to a management device that performs system management functions in place of the central processing unit. It is connected to each functional device via control signal lines, receives status information from each device, and sends configuration commands. However, the controller itself is not located on the data transfer path and does not read, write, or transfer data during transmission. Controller 2 can be, for example, a programmable logic device, such as an FPGA, which uses hardware logic circuits to realize device discovery, status monitoring, and operation management without intervening in the data transmission process between functional devices. Data transmission operation means that data does not flow through controller 2; controller 2 does not transfer or read / write data. The data flow path is provided by interconnect circuits. Data can be directly transferred between these three types of functional devices, for example, via DMA (direct memory access), without passing through controller 2 at all. Interconnector 3 is hardware that provides physical signal paths, featuring multiple device ports, control ports, and corresponding physical links. One side connects to the physical interfaces of various functional devices via the device ports, while the other side connects to the control interface of controller 2 via the control port. After controller 2 completes channel configuration, interconnector 3 internally forms direct physical links between functional devices through the physical links corresponding to the device ports and control ports. Data can be directly transmitted in electrical signal form through these physical links without passing through the pins or cache of the central processing unit. Interconnector 3 establishes direct data transmission physical links between functional devices, supporting direct memory access operations, allowing data to be transmitted directly between functional devices without passing through the central processing unit's cache and registers. During the execution of the target task, the second device, based on the direct memory access function provided by the interconnector, reads the data to be processed from the first device into its local computing space through the interconnector's internal physical links. During the processing of the data to be processed, intermediate processing data is directly written to the third device through the interconnector's internal physical links. After completing the processing of the data to be processed, the processing result is written back to the first device through the internal physical links. The target task refers to the task that requires data interaction between the first device, the second device, and the third device during the execution of the task.

[0017] For example, if the communication protocols of interconnector 3 and each functional device are different, interconnector 3 needs to integrate a protocol conversion module or a protocol conversion processor. For instance, if interconnector 3 uses the PCIe protocol and its internal physical link is a PCIe link, the first device with persistent data storage and the second device with data processing capabilities can be directly connected to the device port as standard PCIe endpoint devices. The third device with online storage capabilities, such as DDR, is based on the DDR bus protocol, which is not a PCIe protocol, and cannot be used as a PCIe endpoint device; protocol conversion is required. For example, the memory can first be connected to the CXL.MEM (Compute Express Link memory) controller, and then the CXL.MEM controller can be connected to the device port. Correspondingly, controller 2 implements the CXLRoot Complex function without CPU intervention. Of course, interconnector 3 can internally integrate a protocol conversion module, placing it within the device port, while the DDR remains connected to the device port. Alternatively, a conventional PCIe memory controller (such as the memory controller built into the FPGA) can be used.

[0018] Taking an example where the first device is an NVMe device, the second device is a GPU, and the third device is memory, with the GPU reading training data from the NVMe disk, intermediate results temporarily stored in memory, and the final result written back to the NVMe disk, the physical link for direct data transmission between the functional devices is as follows: The GPU's DMA engine issues a read request with the target address being the address segment allocated to the NVMe disk in the global address set. This read request data packet enters interconnector 3. Interconnector 3 finds the target port as the port where the NVMe disk is located based on the address and forwards the data packet directly to the NVMe disk. The NVMe disk reads data from the flash memory chip based on the logical block address in the address set and returns the data directly to the GPU memory via interconnector 3. The GPU's DMA engine issues a write request with the target address being the address segment allocated to memory in the global address set. This write request enters interconnector 3, which converts the global address to a physical memory address and writes the data to memory via the DDR bus. Read requests initiated by the GPU pass through interconnector 3, which reads the data corresponding to the physical memory address from memory and returns it to the GPU memory. The GPU sends the computation results as a write request, with the target address being the global address range of the NVMe disk. The write operation is then directly routed to the NVMe disk via interconnector 3. In this embodiment, controller 2 does not participate in data transmission between functional devices; it only manages the operation of each device and controls channel establishment. Meanwhile, interconnector 3, under the control of controller 2, provides a physical link for direct data transmission between the functional devices. All data movement operations in the system are completed directly through the physical link provided by interconnector 3. Data does not need to repeatedly enter and exit the main memory space connected to the central processing unit (CPU), nor does the CPU need to perform read / write processing. This allows a direct data transmission channel to be established between the first device with persistent data storage capabilities, the second device with data computation capabilities, and the third device with online storage capabilities, enabling data interaction without the need for CPU intervention. This reduces data transfer steps and memory read / write operations, lowers data transmission latency, and improves data transmission bandwidth utilization. Since the central processing unit no longer intervenes in the data transfer process, its computing resources and the access bandwidth of the connected main memory are released, and it no longer becomes a factor limiting the data throughput performance between various functional devices. The data interaction rate between various functional devices is improved, thereby improving the data throughput performance between various functional devices, effectively improving the efficiency of user task execution, and meeting the ever-increasing demand for high-throughput and low-latency data transmission.

[0019] Based on the above embodiments, in order to ensure legitimate access between functional devices, this embodiment also includes an access manager connected to the controller 2 and the interconnector 3; the access manager is configured to map the local storage addresses of each functional device to a unified address space and configure access permission information for each functional device.

[0020] The access manager is connected to both controller 2 and interconnector 3. Its function is to perform address mapping and access control for data access between various functional devices. The access manager may have a control interface and a query interface. It can connect to controller 2 through the control interface to receive mapping and permission rules from controller 2, and connect to interconnector 3 through the query interface to receive address query requests from interconnector 3 and return query results. Access permission information includes the access object and / or access address range, indicating the target access device and the allowed address range when each functional device acts as an access request device. For example, the access manager internally includes an address mapping table and a permission configuration table. The address mapping table stores the correspondence between the local storage addresses of each functional device and address segments in the global address space, while the permission configuration table stores the target access device and the allowed address range when each functional device acts as an access initiator.

[0021] In practical applications, the access manager can be implemented by a separate application-specific integrated circuit (ASIC) or an independent logic block within a programmable logic device (PLC). During system initialization, after device discovery, controller 2, based on the local address space information reported by each functional device, divides a non-overlapping address range in the global address space for each functional device and sends this division rule to the address mapping table of the access manager. Simultaneously, controller 2 writes access permission entries between functional devices into the access manager's permission configuration table according to the system's predefined access permission rules. For example, a permission entry can specify that the second device can read and write to the global address range of the first device, the second device can read and write to the global address range of the third device, and the first device can read-only access the global address range of the third device. When the second device initiates a data read from the first device, interconnector 3 intercepts the target address in the access request. This address is a global address, and interconnector 3 sends this global address as a query parameter to the access manager. The access manager looks up the table and finds that the target device corresponding to the global address is the first device, and the corresponding local storage address is the logical block address inside the first device. At the same time, it looks up the permission table and confirms that the second device has read permission for the address range of the first device. Then, it returns the local storage address of the first device to the interconnector 3. The interconnector 3 then routes the access request to the corresponding storage location of the first device to complete the data reading.

[0022] Based on the above features, the unified address mapping provided by the access manager in this embodiment enables each functional device to initiate cross-device access without needing to know the specific physical address of the target device, thus reducing the complexity of address management; the setting of permission configuration information ensures that each functional device can only access target devices and address ranges within the authorized scope, preventing one device from illegally reading or writing to the storage space of other devices, thereby improving the data access security in a multi-device direct connection environment.

[0023] Based on the above embodiments, this embodiment further defines the address mapping and permission verification process of the access manager, which may include the following: The access manager is configured to: construct an address mapping relationship, mapping the storage address of the first device, the computing space address of the second device, and the physical address of the third device to the global address space provided by the interconnector 3; the address mapping relationship represents the correspondence between the local storage address of each functional device and the address segment in the global address space; when an address query request is received from the interconnector 3, the destination global address carried in the address query request is converted into the local storage address of the target access device according to the address mapping relationship; if the access permission information determines that the access requesting device has the access permission of the target access device, the local storage address of the target access device is returned to the interconnector 3.

[0024] The address mapping relationship constructed by the access manager can be a set of lookup table entries indexed by a global address and containing the local storage addresses of each functional device. The storage address of the first device refers to the address code used to locate storage units within the first device, such as the logical block number inside a solid-state drive. The computing space address of the second device refers to the address code of the on-chip storage space directly accessible by the computing units within the second device. The physical address of the third device refers to the storage unit location code formed by the combination of row and column addresses in the storage array of the third device. The global address space provided by the interconnector 3 refers to a unified addressing space maintained by the internal switch of the interconnector 3, spanning all access devices. This space is planned and allocated by the controller 2 based on the address range reported by each device during system initialization and recorded in the address mapping table by the access manager. The access requesting device is the functional device initiating the address query request; the destination global address is the address specified in the address query request; and the target access device is the functional device specified by the access requesting device.

[0025] In this embodiment, when the interconnector 3 receives a cross-device access request from a functional device, the destination global address carried in the request belongs to a certain address value in the global address space. The interconnector 3 sends the destination global address to the access manager for querying. The access manager retrieves the entry corresponding to the destination global address in the address mapping table to obtain the target access device and its local storage address. At the same time, the access manager checks whether the functional device initiating the access has access rights to the target access device in the permission configuration table. If the permission verification passes, the access manager returns the local storage address of the target access device to the interconnector 3, and the interconnector 3 uses the local storage address to complete the data access routing to the target device. For example, the access manager can use a content-addressable storage unit to realize fast lookup of the address mapping table, and the permission configuration table can be implemented using a register array. After receiving the address query request from the interconnector 3, the access manager completes the address mapping table retrieval and permission configuration table retrieval in parallel within one clock cycle. The two retrieval results are matched and judged in combinational logic, and finally outputs a valid address signal or an access denied signal. In addition to the entry-based matching implementation, the access manager can also use an address translation method based on base address plus offset. That is, the global address space is divided into several consecutive address segments according to the functional devices. Each address segment corresponds to the local address start value and address length of a functional device. After receiving the global address, the access manager first determines which address segment it falls into, and then obtains the local storage address of the target device through base address calculation. This method can reduce the storage overhead of the address mapping table.

[0026] Based on the above features, this embodiment completes address translation and permission verification within the same query cycle, reducing the number of interactions between the interconnector 3 and the access manager, and shortening the time from the initiation of the access request to the establishment of the data path; permission verification is embedded in the address translation process, and each cross-device access must pass the permission check, avoiding the possibility of permission check being bypassed.

[0027] Based on the above embodiments, in order to achieve secure access, this embodiment further defines that the access manager assigns device identity identifiers to each functional device and performs authentication, which may include the following: The access manager is also configured to: assign a unique device identity to each functional device, and set access objects and / or access address ranges for the corresponding functional devices based on the device identity; upon receiving functional device authentication information from controller 2, it sets access permission attributes for the authenticated functional devices.

[0028] The device identity identifier is the identification information assigned by the access manager to each connected functional device during the system initialization phase. For example, it can be a set of binary codes, which uniquely corresponds to a functional device within the system. The access permission attribute can be a flag bit associated with each device identity identifier in the access manager's internal permission configuration table. When this flag bit is valid, the corresponding functional device is allowed to participate in cross-device data interaction; when this flag bit is invalid, any cross-device access request from the corresponding functional device will be rejected by the access manager.

[0029] In practical applications, after the system powers on, controller 2 first sends an identity query command to each functional device via the control bus. Each functional device returns its factory-preset device serial number or identification code. Controller 2 then aggregates the collected device information, generates a unique device identity for each device within the system, and sends the correspondence between the device identity and device type / physical parameters to the access manager. The access manager records this correspondence in its internal device identity table and assigns a corresponding device identity and access permission flag to each functional device based on this correspondence, initially in an invalid state. Controller 2 then authenticates each functional device. Authentication methods may include checking if the device serial number is in the authorized device list and checking if the device firmware version meets compatibility requirements. For functional devices that pass authentication, controller 2 notifies the access manager to set the access permission flag corresponding to that device to a valid state. Subsequently, when interconnector 3 forwards a cross-device access request from a functional device to the access manager, the access manager first checks the access permission flag corresponding to the device identity. If the flag is invalid, it directly returns an access denial signal without further address translation and permission matching.

[0030] Based on the above features, in this embodiment, each functional device must undergo authentication before initiating cross-device access. Even if a device that fails authentication is physically connected to the interconnector 3, its access request will be rejected by the access manager because the access permission attribute is not effective, thus preventing unauthorized devices from illegally accessing data within the system.

[0031] Based on the above embodiments, this embodiment further defines the implementation of detecting and blocking abnormal access during data transmission, which may include the following: The access manager is also configured to monitor the data interaction status between various functional devices, and when an abnormal access permission is detected, trigger the generation of an interrupt signal and output the interrupt signal to the interconnector 3 to block the corresponding data transmission channel through the interconnector 3.

[0032] The data interaction status refers to information related to the ongoing data transmission operations between various functional devices, such as the identity of the device initiating the access, the identity of the target device being accessed, the access type, and the access address range. The interrupt signal is a level change signal output by the access manager to interconnector 3 when an anomaly is detected. This signal triggers a reset operation of the internal path selection logic of interconnector 3. The data transmission channel refers to the physical signal path established within interconnector 3 between two functional devices. Blocking the data transmission channel disconnects this physical signal path, preventing the currently transmitted data packet from reaching the target device.

[0033] In practical applications, the access manager can be equipped with a monitoring logic unit. This unit samples the source device identifier, target device identifier, and access address of the cross-device access request currently being processed by interconnector 3 within each clock cycle, comparing them item by item with the allowed access entries in the permission configuration table. The monitoring logic unit can be implemented, for example, using a parallel comparison circuit, completing the matching judgment of all permission entries within one clock cycle. If the current access request does not match any allowed access entry, the monitoring logic unit determines that the access permission is abnormal and immediately sets its internal interrupt signal output pin to an active level. This interrupt signal is connected to the channel control input of interconnector 3 via one or more dedicated signal lines. After detecting the valid interrupt signal, interconnector 3 immediately disconnects the path connection of the channel used by the abnormal access and returns an error status code to the functional device that initiated the access. Simultaneously, the access manager writes abnormal event information, such as the time of the abnormality, the device identifier involved, and the access address, into its internal abnormality record register. Controller 2 can then read these abnormal records through the control interface for fault analysis.

[0034] Based on the above features, this embodiment can detect and block abnormal access behavior during the process of occurrence, shortening the time from the occurrence of the abnormality to the response; the blocking action is completed by the access manager directly sending an interrupt signal to the interconnector 3, without the need for forwarding processing by the controller 2, resulting in a faster response speed and without occupying the processing resources of the controller 2.

[0035] Based on the above embodiments, this embodiment also provides a signal synchronization implementation method, which may include the following: The system also includes a synchronization signal generator connected to each functional device. The synchronization signal generator is configured to send time synchronization signals to multiple types of functional devices and calibrate the local time of each functional device.

[0036] The synchronization signal generator is connected to each functional device and its function is to periodically send time synchronization signals to the functional devices in the system, ensuring that the local clocks of each functional device remain aligned. The synchronization signal generator internally includes a clock source and a signal distribution circuit. The clock source generates a stable periodic pulse signal, and the signal distribution circuit simultaneously drives this pulse signal to the signal output terminals connected to each functional device. Upon receiving the synchronization signal, each functional device compares and adjusts its internal local clock counter with the time information carried in the synchronization signal.

[0037] In practical applications, a synchronization signal generator can be constructed, for example, by a high-precision crystal oscillator and a signal fan-out buffer. The crystal oscillator generates a clock signal at a fixed frequency, which is split into multiple paths by the signal fan-out buffer. Each path is connected to the synchronization signal input pins of the first, second, and third devices via transmission lines of equal length. The synchronization signal generator simultaneously sends a synchronization pulse to all functional devices at fixed time intervals. The edge of the synchronization pulse triggers the reset or increment of the timestamp counter inside each functional device. Besides a dedicated synchronization signal generator implementation, the transmission of the synchronization signal can also be achieved by the clock management unit inside controller 2. Controller 2 periodically sends its own system clock to each functional device through interconnect 3 or the in-band management channel on the control bus. Each functional device extracts time synchronization information at the data link layer and calibrates its local clock.

[0038] Based on the above features, the local time of each functional device in this embodiment is kept aligned by a time synchronization signal. When multiple devices cooperate to send and receive data, they can start and end transmission operations based on a consistent time reference, reducing data transmission conflicts and retransmissions caused by clock deviations.

[0039] Furthermore, the synchronization signal generator is also connected to the controller 2 and is configured to send a time synchronization signal carrying the device identity identifier and the time of the master device to the corresponding slave device according to the device master / slave identity determined by the controller 2 and at a preset time synchronization frequency. The slave device calibrates its local time according to the time of the master device.

[0040] In this context, the master / slave roles are a set of role markers determined by controller 2 during system operation based on the load status and role allocation rules of each functional device. Functional devices designated as master devices assume the role of clock reference providers, while those designated as slave devices receive the master device's time synchronization signal and calibrate their local time accordingly. The master device time is the time information represented by the current count value of the master device's local clock. This time information is encapsulated in the data field of the time synchronization signal and sent along with the synchronization signal.

[0041] In this embodiment, after device discovery and link establishment, controller 2 reads the load status registers of each functional device, selects the second device with a lower current load and clock output capability as the master device, and the remaining functional devices as slave devices. Controller 2 writes the master device's device identity and slave device list into the configuration register of the synchronization signal generator. The synchronization signal generator obtains the master device's current local timestamp through the master device's clock output interface, combines it with the master device's identity to form a time synchronization frame, and sends it to each slave device in the slave device list at a preset time synchronization frequency. Upon receiving the time synchronization frame, the clock calibration circuit inside the slave device first checks whether the master device identity in the frame matches the master device identity previously notified by controller 2. If they match, the local clock counter is updated to the master device time value carried in the frame. If the slave device fails to receive a valid time synchronization signal multiple times consecutively, it reports a synchronization loss event to controller 2, which then decides whether to re-negotiate the master-slave relationship.

[0042] Based on the above features, in this embodiment, the master / slave identity of the device is dynamically determined by the controller 2 according to the system operating status, and the clock reference source can be flexibly switched according to the load. The synchronization signal carries the master device identity identifier, which the slave device can use to determine the source and legality of the synchronization signal, avoiding mistakenly treating other signals as time synchronization signals.

[0043] Based on the above embodiments, this embodiment further defines the internal structure of the interconnector 3, which may include a switch and a bridge; wherein, the switch includes multiple data transmission channels, each data transmission channel corresponding to a set of device interaction links; the bridge isolates the address fields of each functional device and supports each functional device to access the storage space of other functional devices in a direct memory access manner.

[0044] The switch, located within interconnector 3, is responsible for data transmission path selection and signal forwarding. It may contain multiple physical ports, each connected to a functional device via high-speed differential signal lines. Each physical port includes a serial transceiver and a buffer queue. The switch may also contain a path selection matrix, which, based on configuration information from controller 2, electrically connects the transmit / receive paths between any two physical ports, forming a data transmission channel. The bridge is an address domain isolation device located between the switch and each functional device. Its function is to allocate an independent address space for each functional device, ensuring that the address spaces of different functional devices do not overlap. This allows each functional device to use its local address within its own domain when initiating data access, without needing to know the addressing methods of other devices. The bridge also supports direct memory access transmission, meaning that each functional device can initiate direct read / write operations on the storage space of other functional devices through the bridge. Data is directly delivered to the target device's storage unit with the help of the bridge's address translation, bypassing the central processing unit's cache.

[0045] In this embodiment, the switch can be implemented as a PCIe Switch, which provides multiple high-speed differential signal transceiver channels and supports dynamic configuration of channel bandwidth. The bridge can be implemented as an NTB, which is located between the PCIe Switch and each functional device, establishing an independent address translation window for each functional device. During system initialization, controller 2 configures the port mapping table of the PCIe Switch and the address translation table of the NTB through the configuration bus. The port mapping table of the PCIe Switch defines which ports can establish data paths, and the address translation table of the NTB defines the conversion rules between the local address range of each functional device and the internal routing address of the PCIe Switch. After configuration, when the second device initiates a direct memory access read operation on the first device, the read request data packet sent by the second device first passes through the NTB. The NTB converts the destination address in the data packet into a routing identifier used internally by the PCIe Switch. The PCIe Switch forwards the data packet to the port connected to the first device according to the routing identifier. The NTB on the first device side then converts the routing identifier into the local storage address of the first device. After the data is read from the storage unit of the first device, it returns to the second device along the original path.

[0046] As can be seen from the above, this embodiment provides multiple data transmission channels through the switch, enabling data interaction between multiple devices to proceed in parallel without blocking each other; the address domain isolation of the bridge ensures that each functional device maintains its own independent address and does not affect each other; and the support for direct memory access allows data migration to be completed automatically by hardware without software intervention, further reducing the processing overhead during the data migration process.

[0047] Furthermore, the switch in the above embodiment also supports dynamic bandwidth allocation. The switch in this embodiment is also used to monitor the load status of each data transmission channel and adjust the transmission priority and / or virtual channel buffer resource allocation of each data transmission channel according to the load status, so as to balance the data transmission rate of each data transmission channel.

[0048] Link bandwidth refers to the upper limit of the physical layer transmission rate allocated to a data transmission channel in a switch, which can be expressed as bits or bytes transmitted per second. Load status refers to indicators reflecting the busyness of each data transmission channel, such as the number of data transmission requests currently being processed, the amount of data waiting to be transmitted, or the ratio of actual bandwidth used to allocated bandwidth.

[0049] In this embodiment, the switch internally includes a load monitoring circuit and a traffic scheduling circuit. The load monitoring circuit samples the load status of each data transmission channel at fixed time intervals, and the sampled parameters include the fill depth of the buffer queue and the percentage of effective transmission time for each channel. Based on the sampling results of the load monitoring circuit, the traffic scheduling circuit dynamically adjusts the transmission priority of each data transmission channel and / or the allocation of virtual channel (VC) buffer resources.

[0050] Specifically, when the load on a channel exceeds a pre-configured high threshold and persists for a certain number of cycles, the traffic scheduling circuit increases the transmission priority of that channel and increases its corresponding virtual channel buffer queue depth to reduce packet queuing time. Conversely, when the load on a channel falls below a pre-configured low threshold and persists for a certain number of cycles, the traffic scheduling circuit lowers the transmission priority of that channel and releases some virtual channel buffer resources for use by high-load channels. For example, the load monitoring circuit is implemented using a hardware timer and comparator within the switch. The timer triggers sampling every few microseconds, and the comparator compares the sampled value with a pre-configured threshold. When a large amount of data is continuously transmitted on the channel between the first and second devices, while the channel between the second and third devices is idle, the traffic scheduling circuit temporarily allocates the virtual channel buffer resources and port arbitration weights of the idle channel to the high-load channel, giving the high-load channel more frequent port scheduling opportunities and thus increasing its actual data transmission rate. After the high-load transmission is completed, the traffic scheduling circuit restores the original priority and buffer resource configurations of each channel.

[0051] In this embodiment, the above adjustments only involve the traffic scheduling parameters and cache resource allocation within the switch, and do not change the number of physical link channels already established for each channel. The number of physical link channels for each channel is determined by controller 2 during the system initialization phase based on the rated bandwidth requirements of each functional device, and remains fixed during operation.

[0052] Furthermore, the switch also includes a link reconfiguration module and a physical layer multiplexer; the switch is used to: monitor the load status of each data transmission channel, and when the change in the channel load status is detected to meet the preset conditions, trigger the link reconfiguration module to send a link retraining request to the corresponding functional device, and reallocate the physical link channel resources occupied by each data transmission channel through the physical layer multiplexer to adjust the link bandwidth value of each data transmission channel.

[0053] In this embodiment, the switch internally includes a load monitoring circuit, a link reconfiguration module, and a physical layer multiplexer. The load monitoring circuit samples the load status of each data transmission channel at fixed time intervals. When the load status of a certain channel exceeds a pre-configured high threshold and persists for a certain number of cycles, the link reconfiguration module sends a link retraining request to the functional device corresponding to that channel. After the functional device responds, the physical layer multiplexer reallocates the physical link channel resources occupied by each channel, temporarily allocating some physical link channels from idle channels to high-load channels, and negotiating new link width and rate parameters through link retraining. After the bandwidth adjustment is completed, the switch reports the link reconfiguration completion status to controller 2. After the high-load transmission is completed, the link reconfiguration module triggers link retraining again to restore the original channel allocation.

[0054] It should be noted that the above link reconfiguration process involves a brief interruption and re-establishment of the physical layer link state, and is suitable for application scenarios where the requirement for transmission continuity is no higher than the millisecond level of interruption tolerance. For tasks that require continuous and uninterrupted transmission, the switch prioritizes using virtual channel buffer allocation and priority scheduling methods for traffic balancing.

[0055] Based on the above embodiments, this embodiment further defines the establishment and data interaction process of three types of bidirectional data transmission channels between various functional devices, which may include the following: Controller 2 controls interconnector 3 to establish: a first bidirectional data transmission channel between the first device and the second device, a second bidirectional data transmission channel between the second device and the third device, and a third bidirectional data transmission channel between the first device and the third device; the second device reads the data to be processed from the first device into its local computing space through direct memory access, performs calculations on the data to be processed, and directly calls the third device as an extended computing space during the calculation process. After completing the calculation of the data to be processed, the calculation result is directly written to the first device.

[0056] The first bidirectional data transmission channel is a bidirectional physical path established by the interconnector 3 under the control of the controller 2 between the first and second devices. Data can flow from the first device to the second device, and vice versa. This channel allows direct access between the first and second devices without memory intermediaries, as in artificial intelligence tasks. The second bidirectional data transmission channel is a bidirectional physical path between the second and third devices, enabling direct access between them. This is useful for short-term large-data transfer scenarios, where the second device can directly utilize the third device's resources as video memory expansion, improving short-term data processing efficiency. The third bidirectional data transmission channel is also a bidirectional physical path between the first and third devices, allowing direct access between them. This is useful for rapid access and storage of large amounts of data, where data from the first device can be directly written to the third device for rapid access by other devices. The first, second, and third bidirectional data transmission channels operate independently; data transmission on one channel does not consume bandwidth resources from the other two channels, and all three channels can simultaneously transmit data in different directions. The second device reads the data to be processed from the first device into its local computing space via direct memory access. This means that, under the control of its internal direct memory access engine, the second device sends a read request to the interconnector 3 for the storage address of the first device. The interconnector 3 routes this read request to the first device, and the first device directly returns the data from its storage unit to the second device's computing space via the interconnector 3. This entire process requires no instruction intervention from the second device's computing core and does not involve the central processing unit. The extended computing space is used to temporarily store intermediate processing results of the data to be processed. When the second device's local computing space is insufficient, it can use a portion of the third device's storage area as a temporary data storage area. The second device writes the intermediate processing results to the third device through a second bidirectional data transmission channel and reads them back from the third device when needed for subsequent operations. Intermediate processing results refer to the data generated in the previous step during multi-step processing, which serves as input for the next step. This data needs to be temporarily stored somewhere before the final result is generated.

[0057] In this embodiment, during the device discovery and link establishment phases, controller 2 issues three channel establishment instructions to interconnector 3. The switch inside interconnector 3 configures independent port mappings and buffer resources for each channel, and the three channels use different link channel resources at the physical layer. When the second device needs to process the image dataset stored in the first device for model training, the direct memory access engine of the second device initiates continuous read requests to the first device in units of data blocks. The data blocks directly enter the computing space of the second device through the first bidirectional data transmission channel. The second device performs convolution and matrix operations on the data blocks. The amount of intermediate feature map data generated exceeds the remaining capacity of the local computing space. The direct memory access engine of the second device writes the intermediate feature map data to the designated address area of ​​the third device through the second bidirectional data transmission channel. When the operation reaches the subsequent network layer that requires the intermediate feature map, the second device reads back the intermediate feature map data from the third device through the second bidirectional data transmission channel to continue the operation. After all network layer operations are completed, the second device writes the final weights or inference results directly to the result storage area allocated in the first device through the first bidirectional data transmission channel.

[0058] In this embodiment, considering the limited local computing space capacity of the second device in data-intensive computing scenarios, the processing of large datasets may frequently trigger data swapping between local space and external storage due to insufficient space, reducing computing efficiency. This embodiment uses three independent bidirectional channels running in parallel to ensure that data interaction between the three types of devices does not interfere with each other, and each channel can operate at full bandwidth. The second device uses the third device as an extended computing space, expanding the scale of single-process data that can be processed and reducing the data splitting and multiple read / write overhead caused by insufficient local computing space. The computing results are directly written back to the first device, eliminating the step of relaying through the third device and shortening the total time from the start of the computing task to the result being written to disk.

[0059] Based on the above embodiments, this embodiment further defines the device discovery function of the controller 2. In this embodiment, the controller 2 is also configured to: send device discovery messages to each functional device; the device discovery messages carry the system identifier and communication protocol of the functional device; receive device response messages from each functional device; the device response messages include device physical parameters; extract the protocol version field from the device response messages; if the protocol version field of the target functional device matches the corresponding communication protocol, then send a device online notification to the target functional device.

[0060] The device discovery notification is the first communication message sent by controller 2 to each functional device after system startup. This message can be a binary data segment conforming to a preset frame format, containing a system identifier field and a communication protocol field. The system identifier is a unique code generated by controller 2 for this system operation session, used to distinguish the device discovery process of different sessions; the communication protocol field indicates the communication protocol version number supported by controller 2. The device response message is a message sent by each functional device to controller 2 through its own control interface after receiving the device discovery notification. It can be a binary data segment containing the device's physical parameters. The device physical parameters are a set of quantitative indicators describing the hardware capabilities of the functional device. The physical parameters of the first device include at least storage capacity and transmission rate, the physical parameters of the second device include at least computing space capacity and data throughput, and the physical parameters of the third device include at least storage capacity and operating frequency. The protocol version field is a data field in the device response message, indicating the communication protocol version number supported by the functional device. The device online notification is a confirmation message sent by controller 2 to the functional device that has passed the protocol matching check, indicating that the device has been included in the system management scope and can participate in the subsequent data transmission channel establishment.

[0061] In this embodiment, after the system powers on and resets, the startup state machine inside controller 2 enters the device discovery phase. Controller 2 can send a device discovery notification via the control bus. The message frame header contains a fixed synchronization sequence, the frame body contains a system identifier field and a communication protocol version number field, and the frame tail contains a cyclic redundancy check (CRC) code. The interface control circuits inside each functional device continuously monitor the control bus. When a synchronization sequence is detected, it begins to receive complete frames. After successful verification, the frame content is parsed, and the communication protocol version number is compared with the protocol version it supports. Each functional device generates a device response message, which includes its own device type code, protocol version field, and device physical parameter field. The device physical parameters of the first device include at least total storage capacity and sequential read / write rate; the device physical parameters of the second device include at least total computing space capacity and external data throughput bandwidth; and the device physical parameters of the third device include at least total storage capacity and maximum operating frequency. After collecting all device response messages, controller 2 extracts the protocol version field one by one for matching. Functional devices that are consistent with or backward compatible with controller 2's communication protocol version are marked as successful, and device online notifications are sent to these devices one by one. For functional devices with mismatched protocol versions, controller 2 will not send an online notification, and the device will not participate in data interaction during this system operation session.

[0062] In this embodiment, the controller 2 can obtain complete information on the type, capabilities, and protocol compatibility of each access device through a single device discovery process, providing a basis for subsequent channel establishment and resource allocation; the protocol version matching check filters out compatible devices, preventing incompatible devices from participating in subsequent data interactions and causing transmission errors.

[0063] Based on the above embodiments, this embodiment further defines the process of controller 2 establishing a data transmission channel. In this embodiment, controller 2 is also configured to: monitor the load status of each functional device; determine the master device and slave device from each functional device according to the load status of each functional device, and obtain the master / slave identity of the device; control the master device to interact with the corresponding slave device based on the master / slave identity of the device to determine the transmission protocol and data transmission channel; control the interconnector 3 to establish a physical link between the master device and the slave device based on the data transmission channel between the master and slave devices; and receive the link establishment completion signal fed back by the interconnector 3.

[0064] Load status refers to the quantitative representation of the amount of tasks or resource usage currently being processed by each functional device. For example, the load status of a second device may include computing unit utilization, local computing space occupancy, or the depth of the pending task queue; the load status of a first device may include input / output queue depth or read / write operation response time; and the load status of a third device may include storage space occupancy or the number of access conflicts. Preset low-load status conditions refer to quantitative standards for judging a functional device to have a light load, such as computing unit utilization being lower than a pre-configured percentage threshold based on actual business needs, or local computing space occupancy being lower than a pre-configured percentage threshold based on actual business needs. The master device, whose load status meets the preset low-load status conditions, is the functional device that acts as the initiator and coordinator of data transmission in the master-slave relationship. The slave device is the functional device that cooperates with the master device to complete data interaction. Interactive information negotiation refers to the process by which the master and slave devices determine the communication parameters used for subsequent data transmission through the exchange of control messages. Communication parameters include transmission protocol type, channel number, and address window size.

[0065] In this embodiment, the controller 2 may be equipped with a load monitoring module, which periodically reads the status registers of each functional device through the control bus to obtain its load status. The controller 2 compares the load status of each functional device with low load status conditions, and selects one of the functional devices that meets the preset low load status conditions as the master device, with the remaining functional devices as slave devices. The controller 2 notifies the master device of the master-slave identification result. Under the management of the controller 2, the master device and each slave device negotiate information through control messages to determine the transmission protocol version and channel parameters that both parties can support. After the negotiation is completed, the master device reports the negotiation result to the controller 2. Based on this, the controller 2 issues a physical link establishment command to the interconnector 3. After completing the physical link configuration, the interconnector 3 sends a link establishment completion signal back to the controller 2.

[0066] For example, controller 2 can maintain a device load status table internally, recording the load indicators and last update time of each functional device. Controller 2 periodically sends load query commands to each functional device through the management channel on the control bus, and each functional device returns its current load value. Taking a computing scenario with multiple second devices as an example, controller 2 reads the computing unit utilization and remaining local computing space capacity of each second device, selects a second device with a lower computing unit utilization and more remaining local computing space as the master device, and the other second devices, first devices, and third devices as slave devices. Controller 2 sends a master device identity notification message to the selected master device, which contains a list of slave devices and the address information of each slave device. After receiving the notification, the master device initiates a negotiation request to each slave device one by one. The negotiation content includes determining the transmission protocol and data transmission channel parameters. The transmission protocol can select different protocol formats according to the data type and transmission direction. The data transmission channel parameters include the channel number and the maximum data block length for each transmission. After the negotiation is completed, the master device sends the negotiation result to the controller 2. The controller 2 parses the negotiation result and writes the channel parameters into the configuration register of the interconnector 3. The interconnector 3 establishes a physical link according to the contents of the configuration register. After the link is established, the interconnector 3 returns a link ready indication to the controller 2 through the status signal line.

[0067] In this embodiment, the selection of the master device is based on the current load status of each device, avoiding the situation where a high-load device is designated as the master device, which would affect its own computing efficiency; the master and slave devices complete the communication parameter negotiation before the physical link is established, ensuring that both parties can immediately start data transmission with the agreed parameters after the physical link is established, reducing the parameter adjustment overhead after the link is established.

[0068] Based on the above embodiments, this embodiment further defines the process for the controller 2 to migrate the computational load between multiple second devices, which may include the following: Controller 2 is also configured to: monitor the operating status of each second device; when there is a second source device whose computing load exceeds the first load threshold, select the second device that simultaneously meets the requirements of the master device and whose computing load is less than the second load threshold as the second destination device, and issue a task migration instruction to the second source device; in response to the task migration instruction, the second source device saves the current target service's computing context and the status of the processed data as checkpoint data, and directly transmits the checkpoint data and the data to be processed to the second destination device through the interconnector; the second destination device loads the checkpoint data and restores the computing environment, and continues to execute the target service from the breakpoint, thereby migrating the target service of the second source device to the second destination device.

[0069] The first load threshold is an internally set upper limit for computational load in controller 2. When the computational load of a second device exceeds this threshold, the device is considered to be under high load, which may affect computational processing efficiency. The second load threshold is a load threshold lower than the first load threshold, and both can be customized according to actual business needs. When the computational load of a second device is lower than this threshold, the device is considered to have spare computing power to receive migration tasks. The first load threshold is greater than the second load threshold. The second source device is the second device whose computational load exceeds the first load threshold and needs to migrate some or all of its computational tasks. The second destination device is the second device whose current role is the main device and whose computational load is lower than the second load threshold, and which can receive migration tasks. The target service refers to the computational task that needs to be migrated from the second source device to the second destination device, which may include the description information of the data to be processed corresponding to the task, the storage address of the computational program code in the local storage space of the second source device, and the computational context state data.

[0070] In this embodiment, after the controller 2 detects that the computing load of the second source device exceeds the first load threshold, it retrieves the second device currently acting as the main device and whose computing load is below the second load threshold from the device load status table, and selects it as the second destination device. The controller 2 issues a task migration instruction to the second source device, which includes the device identification and communication parameters of the second destination device. The second source device directly transmits the context data and related parameters of its currently pending computing task to the second destination device through the interconnector 3. After the transmission is completed, the second source device reports the migration completion status to the controller 2, and the second destination device receives the task and begins to continue computing processing.

[0071] For example, controller 2 internally sets two configurable load threshold registers to store the values ​​of a first load threshold and a second load threshold, respectively. These thresholds can be set during system initialization based on the rated computing power of each second device. For instance, the first load threshold could be set to 85% of the computing unit utilization, and the second load threshold to 30% of the computing unit utilization. Controller 2 polls the computing load status of each second device at fixed intervals. When it detects that the computing unit utilization of a second device exceeds 85% and persists for multiple sampling cycles, it triggers a load migration process. Controller 2 checks if there is a second destination device in the current system that meets the conditions. If so, it sends a task migration instruction containing the second destination device's identifier and the interconnector 3 port number to the second source device.

[0072] In response to the task migration instruction, the second source device first suspends the issuance of new computation instructions for the current target service and saves the checkpoint data of the current computation task. The checkpoint data includes at least the state of the computation registers, the calculated key tensor data in the video memory, the instruction queue pointer, and the address description information of unprocessed data blocks. The second source device directly transmits the checkpoint data and the data to be processed to the computation space of the second destination device via interconnect 3. After the transmission is complete, the second source device reports the migration completion status to controller 2 and releases the relevant computational resources. The second destination device receives and loads the checkpoint data, restores the same computational environment as the second source device in its local computational space, and continues execution of the target service from the breakpoint. The second source device and the second destination device have the same computational architecture and a compatible instruction set to ensure that the computational context can be directly loaded. If there is no second destination device in the system that meets the conditions, controller 2 keeps the second source device running and checks again in the next sampling period.

[0073] In this embodiment, the computational load is dynamically adjusted among multiple second devices to avoid increased computational latency caused by overload of a single device; the migration destination device is limited to a lightly loaded device that is currently the master device, ensuring that the device receiving the migration task is a device with coordination capabilities and abundant resources in the current system, and the task can start execution immediately after migration.

[0074] Based on the above embodiments, this embodiment further defines the implementation method of extended interconnection of multiple device clusters, which may include the following: Interconnector 3 connects the first device, the second device, and the third device into an independently operating device cluster, and reserves at least one extended interconnect interface for the device cluster. The device cluster connects to other device clusters or external second devices through the extended interconnect interface. Data is transmitted between the second devices connected through the extended interconnect interface in a direct memory access manner to form a computing device cluster.

[0075] In this context, a device cluster refers to an independently operating device unit formed by interconnecting a group of first, second, and third devices together using interconnector 3. Data interaction between the functional devices within this unit is accomplished through physical links provided by interconnector 3, without relying on any devices outside this unit. Extended interconnect interfaces are a set of physical connection ports provided externally by interconnector 3. These ports exist as additional routing nodes within the switches of interconnector 3, and their electrical characteristics and signal protocols are consistent with the ports used to connect functional devices within interconnector 3. A computing device cluster refers to a larger-scale computing combination formed by cascading multiple device clusters through extended interconnect interfaces. The second devices in each device cluster within this combination can directly interact with each other through extended interconnect interfaces. While establishing physical links between internal functional devices, interconnector 3 configures several ports in its switches as extended interconnect interfaces and exposes them externally. These extended interconnect interfaces can be standard high-speed serial interface connectors in physical form. When the extended interconnect interfaces of multiple device clusters are interconnected through cables or backplanes, data between the second devices in each device cluster is transmitted via extended interconnect interfaces using direct memory access. Taking a PCIe switch as an example and a GPU as the second device, the GPU is directly integrated into the memory and NVMe through the PCIe switch to form an independently operating device assembly. PCIe channels can be reserved for external use, and multiple GPUs can be interconnected and expanded to form a GPU cluster.

[0076] For example, the internal switch of interconnector 3 in a device cluster provides sixteen downlink ports for connecting functional devices, and four uplink ports as extended interconnect interfaces. These four uplink ports are connected to the external interface panel of the device cluster via connectors. When a computing device cluster needs to be built, high-speed cables are used to connect the uplink ports of one device cluster to the uplink ports of another device cluster one by one. After detecting the establishment of the uplink port link, controller 2 sends a cluster discovery message to controller 2 of the peer device cluster through the extended interconnect interface, exchanging device capability information and address mapping tables between the two device clusters. The merging of the address mapping tables is completed collaboratively by the access managers of the two device clusters. Each device cluster's global address space is reallocated a high-order address prefix to distinguish different clusters. After the address mapping tables of the two clusters are merged, the second device in each cluster can directly access the functional devices within the peer cluster using a global address with the address prefix of the peer cluster. Data transmitted on the extended interconnect interface uses the same direct memory access method as data interaction within the cluster. When the second device initiates a data migration request to the second device in the remote cluster, the data travels from the local computing space of the source second device through the source interconnect, the extended interconnect interface, and the destination interconnect to finally reach the local computing space of the destination second device, without passing through the central processing unit.

[0077] In this embodiment, the second devices within the computing device cluster can directly interact with each other, and the data does not need to be relayed through the central processing unit. During large-scale parallel computing, the second devices can efficiently share intermediate data, which greatly improves the efficiency of large-scale data processing. The number of devices in the cluster can be flexibly expanded according to computing needs. The expansion interface uses a standard high-speed serial protocol, which is compatible with second devices from different manufacturers and models.

[0078] For example, as a simpler implementation, controller 2 can consist of a programmable logic device and its configuration storage unit. This programmable logic device can be configured with device status monitoring logic, channel configuration logic, clock management logic, and address mapping logic, such as... Figure 2As shown. In practical applications, the implementation logic of the controller, access manager, and synchronization signal generator in the above embodiments is integrated on a single FPGA board, which can be used as the controller for the entire system. After the system powers on, the FPGA sends device discovery signals to each functional device sequentially via the control bus. Each functional device returns device type information and capability parameters through its own control interface, records the returned information in its internal device status table, and completes the device online confirmation. Subsequently, the data transmission paths that need to be established between each device are determined according to the device type. For example, it is determined that bidirectional data paths need to be established between the first and second devices, between the second and third devices, and between the first and third devices. The FPGA sends a channel establishment command to the interconnector 3, which carries the port numbers of the source and destination devices. The interconnector 3 configures its internal path selection logic according to the command and connects the corresponding ports at the electrical level to form a physical link. After the physical link is established, the interconnector 3 returns a link ready signal to the FPGA. The FPGA constructs an address mapping relationship, mapping the storage address of the first device, the computing space address of the second device, and the physical address of the third device to the global address space provided by interconnector 3. When an address query request is received from the interconnector, the global address carried in the address query request is converted into the local storage address of the target access device according to the address mapping relationship. If the access permission information determines that the access requesting device has the access permission of the target access device, the local storage address of the target access device is returned to the interconnector. Subsequently, the second device can directly initiate a data read operation to the first device through interconnector 3, and the first device directly transmits the stored data to the local computing space of the second device through interconnector 3. The central processing unit does not participate in the data reading, writing, or movement during the entire process. One implementation of interconnector 3 consists of a switching circuit and a bridging circuit. The switching circuit provides multiple physical ports, each port is connected to a functional device through a high-speed differential signal pair. The switching circuit contains a path selection matrix, which connects the transmit and receive signal lines between any two ports according to the configuration information issued by the FPGA. A bridging circuit is located between the switching circuit and each functional device. It assigns an independent address domain to each functional device, allowing each device to use its local address within its respective domain when initiating data access. The bridging circuit performs address domain conversion during transmission, thus avoiding address conflicts. For example, the first device might use a logical block address range, the second device a local storage address range, and the third device a physical address range. These address domains are independent of each other. The bridging circuit records the address domain correspondence when the physical link is established. When a data packet passes through the bridging circuit, its address field is replaced with the target device's local address, thereby enabling direct data transfer across address domains.

[0079] As can be seen from the above, this embodiment bypasses the CPU relay, significantly reducing data transmission latency and improving the overall system operating efficiency. The data transmission rate between the first and second devices is effectively improved, and latency is reduced. By constructing a full-link direct connection architecture through switching and bridging circuits, bidirectional direct connection and parallel transmission between NVMe, GPU, and memory are achieved, fully leveraging the hardware performance of each device.

[0080] Finally, the present invention also provides a server, please refer to [link to relevant documentation]. Figure 3 The server includes the data interaction system described in any of the above embodiments.

[0081] The server is a complete machine containing basic computing resources such as a central processing unit (CPU), system memory, power supply, and heat dissipation components. The CPU runs the operating system and applications, performs system management, and schedules tasks. In this embodiment, the multi-device data interaction system exists as a subsystem within the server. This subsystem is connected to the CPU via a management bus. The CPU can issue startup commands and operating parameters to the controllers within the subsystem via the management bus, but data interaction between the functional devices within the subsystem does not go through the CPU. Physically, the components of the multi-device data interaction system are installed inside the server chassis and operate using the power and cooling provided by the server.

[0082] For example, a motherboard is installed inside the server chassis, on which the central processing unit (CPU) and system main memory are soldered or plugged in. The interconnectors, access managers, and synchronization signal generators of the multi-device data interaction system are integrated on an expansion board, which connects to corresponding interfaces on the motherboard via high-speed connectors. The first device is installed on the server's storage bay and connected to the corresponding port on the expansion board via a cable. The second device is installed in the server's expansion card slot and connected to the corresponding signal traces on the expansion board via the slot's gold fingers. The third device is installed in the server's memory card slot and connected to the expansion board via the slot. The controller is installed on the expansion board and interconnects with the interconnectors, access managers, and synchronization signal generator via onboard traces, and connects to the CPU on the motherboard via a management bus interface. After the system powers on, the CPU sends a startup command to the controller via the management bus. The controller then independently completes device discovery, channel establishment, and operational management. Data interaction between the various functional devices is completed within the interconnectors on the expansion board, without going through the system main memory and CPU on the motherboard.

[0083] In this embodiment, data transmission between the computing acceleration device and the storage device within the server does not require CPU resources, allowing the CPU to focus on other computing tasks. The overall data processing throughput of the server is not limited by the bus bandwidth between the CPU and main memory, thus improving the overall processing efficiency in data-intensive application scenarios.

[0084] Finally, this embodiment takes an NVMe device as the first device, a GPU as the second device, and memory as the third device, with the user task being a large model training task. The server consists of a GPU, NVMe hard drive, DDR5, FPGA, PCIe 5.0 switch, and NTB (Non-Transparent Bridging). The FPGA integrates controller 2, access manager, and synchronization signal generator. The PCIe switch integrates the NTB, and one or more dedicated device ports integrate a CXL.MEM controller. This PCIe switch is used as an interconnect 3, and the dedicated device ports are used to connect to the DDR5. Figure 4 As shown, it may include the following: After the server starts, Controller 2 sends a device discovery message via the PCIe bus. This message carries the system identifier and the communication protocol used for interaction. The GPU and NVMe respectively send back corresponding device response messages through their respective processors. The device response messages include service information such as the GPU's memory capacity and bandwidth, the NVMe's storage capacity and transfer rate, and the memory's capacity and frequency. Controller 2 parses the returned device response messages, confirms that the device supports the communication protocol, and notifies the device to go online after confirmation, thus completing device identification. Controller 2 negotiates master-slave device relationships based on device load. If the GPU is under low load, it is determined as the master device, and the NVMe and memory are the slave devices. It controls the GPU to negotiate and interact with the NVMe and memory, allocates data transmission channels, and controls the interaction links between the GPU and memory, and between the GPU and NVMe. The transmission protocol is NVMe over PCIe, and a physical link is established with the PCIe switch via NTB. After the link is established, a feedback signal is sent to Controller 2.

[0085] The access manager can, for example, use an ATS (address translation service) to map the physical addresses of NVMe storage, GPU memory, and RAM to the PCIe global address space. It assigns identity identifiers to each device; for example, GPU's identifier is 0x01, NVMe's is 0x02, and RAM's is 0x03. Controller 2 grants access permissions after successful verification. Based on the established links, it configures the GPU to access the full address spaces of NVMe and RAM, NVMe to access designated areas of GPU memory and RAM, and RAM to access designated areas of GPU memory and the full address space of NVMe, prohibiting unauthorized access. The GPU, as the master device, sends a time synchronization signal at fixed intervals, carrying a timestamp and identity identifier. Upon receiving this signal, NVMe and RAM calibrate their local clocks. After the model training task starts, the GPU reads relevant data from NVMe to GPU memory via DMA operation and links. The GPU performs calculations on the data. During the process, it can directly call memory as extended video memory to temporarily store intermediate data results. After the processing is completed, the GPU writes the final result directly to NVMe. The whole process does not involve the CPU, and the data transmission latency is controlled.

[0086] During task execution, Controller 2 also monitors the operating status of each device in the server. If the computing load of a certain GPU increases to 90% or higher, it automatically performs load balancing, evenly distributing tasks to other master devices or directly limiting the load to ensure the stable operation of the master devices. It monitors link data via the PCIe Switch. When the data transfer demand between NVMe and memory increases, it adjusts the PCIe channel bandwidth of both devices to expand the link bandwidth between NVMe and memory, improving transmission efficiency. If abnormal memory address access is detected, it triggers an interrupt through the access manager, blocking the relevant access and reporting it to ensure server security.

[0087] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be performed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The systems, computing devices, or apparatuses described herein encompass a wide range of means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0088] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0089] The foregoing has provided a detailed description of a multi-device data interaction system and server provided by the present invention. The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Several improvements and modifications can be made to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A multi-device data interaction system, characterized in that, include: Multifunctional devices include at least a first device with persistent data storage function, a second device with data processing function, and a third device with online storage function; The controller is connected to each functional device and is configured to: not participate in data transmission between functional devices, but only manage the operation of each functional device and control the establishment of data transmission channels between functional devices; An interconnector has multiple device ports, a control port, and physical links corresponding to each device port and control port. The first device, the second device, and the third device are respectively connected to each device port, and the controller is connected to the control port. Under the control of the controller, the interconnector provides physical links for direct data transmission to each functional device, enabling data interaction between the devices without intermediary processing by a central processing unit. During the execution of a target task, the second device, based on the direct memory access function provided by the interconnector, reads the data to be processed from the first device into its local computing space via the interconnector's internal physical link. During the processing of the data to be processed, intermediate processing data is directly written to the third device via the interconnector's internal physical link. After completing the processing of the data to be processed, the processing result is written back to the first device via the internal physical link.

2. The multi-device data interaction system according to claim 1, characterized in that, It also includes an access manager connected to the controller and the interconnector; The access manager is configured to map the local storage addresses of each functional device to a unified address space and configure access permission information for each functional device. The access permission information includes the access object and / or the access address range, which indicates the target access device and the range of addresses that are allowed to be accessed when each functional device acts as an access request device.

3. The multi-device data interaction system according to claim 2, characterized in that, The access manager is configured to: construct an address mapping relationship, and through the address mapping relationship, map the storage address of the first device, the computing space address of the second device, and the physical address of the third device to the global address space provided by the interconnector; the address mapping relationship represents the correspondence between the local storage address of each functional device and the address segment in the global address space; When an address query request is received from the interconnector, the destination global address carried in the address query request is converted into the local storage address of the target access device according to the address mapping relationship; If the access request device is determined to have the access permission of the target access device based on the access permission information, the local storage address of the target access device is returned to the interconnector.

4. The multi-device data interaction system according to claim 2, characterized in that, The access manager is also configured to: Assign a unique device identifier to each functional device, and set access objects and / or access address ranges for the corresponding functional devices based on the device identifiers; Upon receiving the functional device authentication information from the controller, the access permission attribute is set for the successfully authenticated functional device.

5. The multi-device data interaction system according to claim 2, characterized in that, The access manager is also configured to: The system monitors the data interaction status between various functional devices. When an abnormal access permission is detected, it triggers the generation of an interrupt signal and outputs the interrupt signal to the interconnector to block the corresponding data transmission channel through the interconnector.

6. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, It also includes a synchronization signal generator connected to each functional device, the synchronization signal generator being configured as follows: Send time synchronization signals to various functional devices to calibrate the local time of each device.

7. The multi-device data interaction system according to claim 6, characterized in that, The synchronization signal generator is also connected to the controller and is configured as follows: Based on the master-slave identity determined by the controller, a time synchronization signal carrying the master device's device identity identifier and master device time is sent to the corresponding slave device at a preset time synchronization frequency. The slave device then calibrates its local time according to the master device's time.

8. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, The interconnect includes switches and bridges; The switch includes multiple data transmission channels, each corresponding to a set of device interaction links; The bridge isolates the address domains of each functional device and allows each functional device to access the storage space of other functional devices via direct memory access.

9. The multi-device data interaction system according to claim 8, characterized in that, The switch is also used to: monitor the load status of each data transmission channel, and adjust the transmission priority and / or virtual channel buffer resource allocation of each data transmission channel according to the load status, so as to balance the data transmission rate of each data transmission channel.

10. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, The controller controls the interconnect to establish: a first bidirectional data transmission channel between the first device and the second device, a second bidirectional data transmission channel between the second device and the third device, and a third bidirectional data transmission channel between the first device and the third device; the first bidirectional data transmission channel, the second bidirectional data transmission channel, and the third bidirectional data transmission channel operate independently; The second device reads the data to be processed from the first device into its local computing space via direct memory access, performs calculations on the data to be processed, and directly calls the third device as an extended computing space during the calculation process. After completing the calculation of the data to be processed, the calculation result is directly written to the first device. The extended computing space is used to temporarily store the intermediate processing results of the data to be processed.

11. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, The controller is also configured to: Send device discovery messages to each functional device; the device discovery messages carry the system identifier and communication protocol of the functional devices; Receive device response messages from each functional device; the device response messages include device physical parameters; Extract the protocol version field from the device response message. If the protocol version field of the target functional device matches the corresponding communication protocol, send a device online notification to the target functional device. The physical parameters of the first device include at least storage capacity and transmission rate; the physical parameters of the second device include at least computing space capacity and data throughput; and the physical parameters of the third device include at least storage capacity and operating frequency.

12. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, The controller is also configured to: Monitor the load status of each functional device; Based on the load status of each functional device, the master device and slave device are determined from each functional device to obtain the master and slave device identities; wherein, the load status of the master device meets the preset low load status condition; Based on the master-slave identity of the devices, the master device is controlled to interact with the corresponding slave device to determine the transmission protocol and data transmission channel; Based on the data transmission channel between the master and slave devices, the interconnector is controlled to establish a physical link between the master device and the slave device; Receive the link establishment completion signal fed back by the interconnector.

13. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, The second device may be multiple, and the controller is further configured to: Monitor the operational status of each secondary device; When there is a second source device whose computing load exceeds the first load threshold, select the second device that simultaneously meets the identity of the master device and whose computing load is less than the second load threshold as the second destination device, and migrate the target service of the second source device to the second destination device. Wherein, the first load threshold is greater than the second load threshold.

14. The multi-device data interaction system according to any one of claims 1 to 5, characterized in that, The interconnector connects the first device, the second device, and the third device into an independently operating device cluster, and reserves at least one extended interconnection interface for the device cluster. The device cluster connects to other device clusters or external second devices through the extended interconnect interface. The second devices connected through the extended interconnect interface transmit data in a direct memory access manner to form a computing device cluster.

15. A server, comprising a central processing unit, characterized in that, It also includes the multi-device data interaction system as described in any one of claims 1 to 14.

Images