Data transmission method and system, device, storage medium and program product

Abstract

The embodiments of the present application provide a data transmission method and system, device, storage medium and program product, which relate to the field of storage technology, including: in a first controller, a first data set is sent to a data processing module, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; when a preset condition is met, the data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, wherein the number of second data sets is the same as the number of data storage devices; the second data set is sent to a second controller through a transmission channel corresponding to the data storage device. The above scheme is adopted to solve the technical problem of low data transmission efficiency between controllers in the current storage system in the related technology.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of storage technology, and specifically, to a data transmission method and system, device, storage medium and program product. Background Art

[0002] In a centralized storage system, the storage architecture can be implemented using hardware such as servers, controllers, and hard disk cabinets. Multiple controllers can be supported to improve availability and performance. The head can connect servers and disk array devices to provide high-speed data transmission. The head can manage disks, abstract disks into storage resource pools, and divide logical unit numbers for client use.

[0003] In the related art, after the head receives the I / O request sent by the server, the I / O request can be processed to convert the logical address in the I / O request into a physical address and transmit the data to the disk array at the back end. In each data transmission, a certain communication protocol needs to be followed, including establishing a connection, transmission control, error detection, etc. However, the above process needs to be performed for I / O requests of different sizes. In the process of transmitting small I / O requests, these protocol overheads account for a larger proportion of the transmitted data, resulting in lower data transmission efficiency under the same bandwidth.

[0004] In the related art, an effective solution has not yet been proposed for the technical problem of low data transmission efficiency between controllers in current storage systems. Summary of the invention

[0005] The embodiments of the present application provide a data transmission method and system, an apparatus, a storage medium, and a program product to at least solve the technical problem of low data transmission efficiency between controllers in current storage systems in the related art.

[0006] According to an embodiment of the present application, a data transmission method is provided, including: in a first controller, sending a first data set to a data processing module, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; when a preset condition is met, the data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, wherein the number of second data sets is the same as the number of data storage devices; and sending the second data sets to a second controller through a transmission channel corresponding to the data storage device.

[0007] In an exemplary embodiment, when the preset conditions are met, the data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, including: in the data storage device, aggregating the multiple first data blocks included in the stored second data set to obtain aggregated second data blocks, and using the second data blocks to update the second data set to obtain an updated second data set.

[0008] In an exemplary embodiment, the above-mentioned aggregating the multiple first data blocks included in the stored second data set includes one of the following: aggregating the multiple first data blocks included in the second data set stored in the data storage device into one second data block; aggregating the multiple first data blocks included in the second data set stored in the data storage device into multiple second data blocks, wherein the data size of the second data block is greater than the preset data size.

[0009] In an exemplary embodiment, the above-mentioned data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, including: sending the multiple second data sets obtained by splitting to the data storage device connected to the chip through the chip of the data processing module, wherein the number of chips of the data processing module is consistent with the number of data storage devices.

[0010] In an exemplary embodiment, when the preset conditions are met, the data processing module splits the first data set into multiple second data sets, including one of the following: when the sum of the sizes of the first data blocks included in the first data set is greater than a preset data threshold, the first data set is split into multiple second data sets; the first data set composed of the first data blocks received within the first time is split into multiple second data sets.

[0011] In an exemplary embodiment, the data processing module splits the first data set into multiple second data sets, including: determining the total size of the first data blocks included in the first data set, and determining the mean data size based on the total size of the first data blocks and the number of data storage devices; adding the first data blocks included in the first data set in sequence as the i-th second data set until the total size of the first data blocks included in the i-th second data set is greater than the mean data size, thereby determining the i-th second data set; after determining the p-1-th second data set, determining the remaining first data blocks as the p-th second data set, where i is a positive integer greater than 0 and less than p, and p is the number of data storage devices.

[0012] In an exemplary embodiment, the above-mentioned sending the second data set to the second controller through the transmission channel corresponding to the data storage device includes: sending the second data set to the bridge port; sending the second data set to the second controller through the bridge port and the transmission channel, wherein the transmission channel is connected to the data processing module in the second controller.

[0013] In an exemplary embodiment, the sending of the second data set to the bridge port includes: when the second data set meets a sending condition, the memory access module sends the second data set in the data memory to the bridge port.

[0014] In an exemplary embodiment, the sending of the second data set to the bridge port includes: mapping a sending address of the second data set to obtain a receiving address corresponding to the second controller.

[0015] In an exemplary embodiment, before sending the first data set to the data processing module, the step includes: acquiring the first data set at an input port.

[0016] According to another embodiment of the present application, a data transmission system is provided, including: a first controller, wherein the first controller includes a data processing module, a data storage device, a first PCIe switch, and an input port, and the first PCIe switch is used to implement multi-channel data transmission with a second PCIe switch of the second controller; and a second controller, wherein the second controller includes a second PCIe switch, and the second PCIe switch is used to implement multi-channel data transmission with the first PCIe switch.

[0017] In an exemplary embodiment, the first PCIe switch includes multiple partitions, and the uplink port of each partition of the first PCIe switch is connected to the data processing module, wherein the uplink port is used to transmit data with the data processing module; the downlink port of each partition of the first PCIe switch is connected to the downlink port of the corresponding partition of the second PCIe switch, wherein the downlink port is used to realize data transmission between the partition of the first PCIe switch and the corresponding partition of the second PCIe switch.

[0018] In an exemplary embodiment, the uplink port of each partition of the first PCIe switch is respectively connected to the corresponding chip in the data processing module, wherein the number of partitions of the first PCIe switch is the same as the number of chips, and the partitions of the first PCIe switch correspond one-to-one to the chips.

[0019] In an exemplary embodiment, each partition of the second PCIe switch is respectively connected to a corresponding chip in a data processing module of the second controller, wherein the number of partition data of the second PCIe switch is the same as the number of chips of the second controller, and the partitions of the second PCIe switch correspond one-to-one to the chips of the second controller.

[0020] In an exemplary embodiment, the chips of the above-mentioned data processing module are interconnected via an off-chip bus, and the chips are respectively connected to corresponding data storage devices, wherein the data storage devices are used to store data blocks.

[0021] In an exemplary embodiment, the first PCIe switch includes a plurality of DMA engines, the number of the DMA engines being the same as the number of the chips, and the DMA engines are used to fetch data from corresponding data storage devices.

[0022] In an exemplary embodiment, the first PCIe switch and the second PCIe switch are connected via a non-transparent bridge port, and the non-transparent bridge port is used to translate the data address in the first PCIe switch into the address of the second PCIe switch or translate the address of the second PCIe switch into the address of the first PCIe switch.

[0023] In an exemplary embodiment, the input port is connected to a server and is used to receive data sent by the server.

[0024] According to another embodiment of the present application, a data transmission device is provided, including: a first sending module, used to send a first data set to a data processing module in a first controller, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; a splitting module, used to split the first data set into multiple second data sets by the data processing module when preset conditions are met, and send the multiple second data sets to corresponding data storage devices respectively, wherein the number of second data sets is the same as the number of data storage devices; a transmission module, used to send the second data set to the second controller through a transmission channel corresponding to the data storage device.

[0025] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0026] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0027] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method in each embodiment of the present application are implemented.

[0028] Through the present application, in the first controller, the first data set is sent to the data processing module, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; when the preset conditions are met, the data processing module splits the first data set to obtain multiple second data sets, and sends the multiple second data sets to the corresponding data storage, wherein the number of the second data sets is the same as the number of the data storage; the second data set is sent to the second controller through the transmission channel corresponding to the data storage. In the first controller, after obtaining the first data set, the first data set can be sent to the data processing module, so that the first data set is split in the data processing module to obtain multiple second data sets, and the second data set can be sent from the data processing module to the corresponding data storage. The second data set in the data storage can be sent to the second controller through the corresponding transmission channel. In order to realize the parallel sending of the first data set, the efficiency of data transmission is improved, thereby solving the technical problem of low data transmission efficiency between controllers in the current storage system in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a hardware structure block diagram of an optional data transmission method according to an embodiment of the present application;

[0030] Figure 2 is a flowchart of an optional data transmission method according to an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of an optional data transmission method according to an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of another optional data transmission method according to an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of another optional data transmission method according to an embodiment of the present application;

[0034] Figure 6 is a schematic diagram of another optional data transmission method according to an embodiment of the present application;

[0035] Figure 7 is an architecture diagram of an optional data transmission system according to an embodiment of the present application;

[0036] Figure 8It is a structural block diagram of an optional data transmission device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0039] Figure 1 1 is a hardware structure diagram of an optional data transmission method of an embodiment of the present application. Figure 1 As shown in the figure, a centralized storage area network (SAN) is a dedicated storage network that can connect servers and storage devices through high-speed fiber channels. Figure 1 As shown, the controller is the core component of the storage system, including two controllers (controller 1 114 and controller 2 122). The controller can be connected to server 1 102, server 2 104, etc. through a fiber optic switch 106. Figure 1 The number of servers is only an example, and the number of servers is not limited. Different controllers can be connected through a communication bus 124 to achieve data intercommunication between controllers, and the controller can be connected to a disk array 126 to implement read / write instructions of the server.

[0040] The fiber switch 106 may be a high-speed fiber channel (such as FC or RoCE) to connect the controller and the server. The disk array 126 may be a physical device for storing data, and multiple disks may be connected in series to achieve data redundancy and fault tolerance. The controller may implement various functions in a centralized SAN storage system, such as disk management, disk abstraction into a storage resource pool, and division of logical unit numbers for client use.

[0041] Optionally, controller 1 114 may include front-end host interface 1 108, cache 1 110, back-end bus 1 112, and controller 2 122 may include front-end host interface 2 116, cache 2 118, back-end bus 2 120, etc. Data transmission is achieved between different controllers via communication bus 124. The front-end host interface is the part for the controller to communicate with the server, which can process I / O requests from the server and convert these requests into operations on the disk array. The front-end interface can be of different interface types (such as PCIe), depending on the requirements of the connected server and disk array, which can enable data to flow from the server to the disk array, or from the disk array to the server.

[0042] Cache is a fast storage resource used to temporarily store data that is about to be written to a disk array (write cache) or read from a disk array (read cache). Cache can significantly increase data access speed because it is much faster than the access speed of the disk array. The use of cache can reduce the number of accesses to the storage medium, thereby improving overall performance.

[0043] The back-end bus, which can be a back-end interface, is a channel inside the controller for connecting the cache and the disk array, and can transmit data between the cache and the disk array.

[0044] Optionally, in a centralized storage system, the two controllers can be designed to back up each other to improve the availability and fault tolerance of the system. The two controllers can work in the following modes: Active-Passive mode: In this mode, usually only one controller (the active controller) handles all I / O operations and storage management tasks, while the other controller (the standby controller) is in standby state, waiting to take over when the active controller fails. This mode ensures that when the active controller fails, the standby controller can quickly take over, thereby achieving seamless switchover and minimizing service interruption. Active-Active mode: In this mode, both controllers participate in storage operations, and they can process I / O requests simultaneously, thereby improving the overall performance of the storage system. The active-active configuration can provide load balancing and higher throughput because the two controllers can work in parallel to share the I / O load. Load Sharing mode: The two controllers can share different storage tasks or manage different storage resource pools, which can optimize resource utilization and improve efficiency.

[0045] Optionally, no matter which working mode the two controllers are in, when one controller fails, the other controller should be able to automatically take over the tasks of the failed controller to ensure the continuous operation of the storage system, that is, to achieve failover. Therefore, it is necessary to maintain the consistency of the cache of the two controllers to ensure the accuracy and consistency of the data, that is, data synchronization can be performed between the two controllers, and the active controller can synchronize the status to the standby controller.

[0046] In an optional implementation, when data is transmitted between two controllers, after the main controller receives the data sent by the server, it can store the data in the cache and synchronously send the data in the cache to the other controller to achieve data synchronization between the controllers. In the process of transmitting data from the main controller to the other controller, the data can be split and the split data can be sent to the other controller through multiple channels to achieve efficient data transmission.

[0047] Through the present application, in the first controller, the first data set is sent to the data processing module, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; when the preset conditions are met, the data processing module splits the first data set to obtain multiple second data sets, and sends the multiple second data sets to the corresponding data storage, wherein the number of the second data sets is the same as the number of the data storage; the second data set is sent to the second controller through the transmission channel corresponding to the data storage. In the first controller, after obtaining the first data set, the first data set can be sent to the data processing module, so that the first data set is split in the data processing module to obtain multiple second data sets, and the second data set can be sent from the data processing module to the corresponding data storage. The second data set in the data storage can be sent to the second controller through the corresponding transmission channel. In order to realize the parallel sending of the first data set, the efficiency of data transmission is improved, thereby solving the technical problem of low data transmission efficiency between controllers in the current storage system in the related technology.

[0048] In this embodiment, a data transmission method is provided. Figure 2 is a flowchart of an optional data transmission method according to an embodiment of the present application, such as Figure 2 As shown, the data transmission method includes:

[0049] Step S202, in the first controller, sending the first data set to a data processing module, wherein the first data set includes a plurality of first data blocks, and the data processing module is used to process the data blocks;

[0050] It should be noted that the first controller may be a controller in a different working state, such as a main controller or any one of two controllers that are both performing storage operations.

[0051] It should be noted that the first data set may be a set of first data sent by the server to the controller, and the first data may be a read or write request sent by the server, which may include the location of the data to be accessed (such as a logical unit number, a file path or a block address) and the content of the data to be transmitted. The first data may also be a control command sent by the server to the controller to instruct the controller to perform specific operations, such as creating or deleting a file system, executing a snapshot, adjusting the volume size, etc. The first data may also be metadata, application data, database data, management commands, etc. related to file system operations. There is no limitation on the specific form of the first data.

[0052] In an optional implementation, the server may send multiple first data to the controller. The first data set may include multiple first data blocks. The first data block may be a unit for storing data in the file system. The size of the first data block is not limited and may be 512 bytes, a small I / O request such as 4KB, or a large I / O request such as 8KB, 16KB, or 1MB.

[0053] In an optional implementation, the controller may include a data processing module, and the data processing module may process the first data set, such as managing the transmission of the first data set between the cache and the disk array.

[0054] Step S204: when the preset conditions are met, the data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, wherein the number of the second data sets is the same as the number of the data storage devices;

[0055] It should be noted that the data processing module can split the first data set to obtain multiple second data sets. Splitting the first data set is to regroup the first data blocks in the first data set to obtain multiple new sets without changing the first data blocks.

[0056] It should be noted that the data processing module can be the CPU of the controller, which can process the instructions sent by the server and the sent data, which can be connected to the cache in the controller and can send the received first data set to the cache. The data processing module can split the first data set, and the splitting rules can be pre-set, such as the number of second data sets obtained by splitting.

[0057] In an optional implementation, after splitting the first data set to obtain multiple second data sets, the multiple second data sets can be sent to a data storage device, which is a cache of the controller. The data storage device can be used to temporarily store frequently accessed data to reduce the number of accesses to the main memory, thereby improving data access speed and overall system performance. For a first data block of a write data type, the first data block can be first written to the data storage device and then asynchronously written to the main memory (such as a disk array).

[0058] In an optional implementation, the data processing module of the controller may be connected to a plurality of data storage devices, and the number of the second data sets may be the same as the number of the data storage devices, that is, one second data set may be stored in each data storage device.

[0059] Step S206: Send the second data set to the second controller through the transmission channel corresponding to the data storage.

[0060] It should be noted that the data storage device may physically have a corresponding transmission channel, and the transmission channel may be a data transmission channel constructed using the PCIe bus standard. The two controllers may be used as two PCIe devices, and each controller may include a PCIe switch. The switch may forward data in a controller from one port to another port, thereby realizing data transmission between the two controllers.

[0061] It should be noted that the transmission channel can be a channel that can realize data transmission by interconnecting two controllers based on high-speed data transmission technologies such as PCIe bus, Infiniband or RoCE network. Different transmission technologies can be used to build transmission channels for different application scenarios. PCIe bus can be used to connect between hardware components and can support point-to-point connection; Infiniband can connect servers and storage clusters and can ensure the reliability of data transmission between hardware, but its cost is relatively high; RoCE network is a remote direct memory access technology based on Ethernet, and RoCE protocol has two versions. Compared with PCIe technology, Infiniband has the advantage of lower cost, and compared with RoCE network, it has the advantages of no protocol conversion and low latency. Therefore, PCIe bus can be used as the preferred solution for building a transmission channel for data synchronization between controllers.

[0062] In an optional implementation, PCIe interfaces may be provided in the two controllers, and the PCIe interfaces may be an upstream port (USP) and a downstream port (DSP). A PCIe switch may be provided in each controller, and the first controller and the second controller may be connected by using the PCIe switch. The connection method may be to physically connect the PCIe ports of the two controllers, and configure the PCIe devices by software to achieve management and identification. How to use the transmission channel to achieve data transmission will be described in detail below and will not be repeated here.

[0063] It should be noted that the data memory can be different types of memory, such as DDR, LPDDR, SRAM, NAND Flash, ROM, etc.

[0064] In an optional implementation, the second data set may be sent to the second controller via a transmission channel corresponding to the data storage device, and a corresponding transmission channel may be configured for each data storage device to send the second data set to the second controller via the corresponding transmission channel. The transmission channel corresponds to the data storage device, and each data storage device may have an independent transmission channel. How to establish an independent transmission channel will be described in detail below and will not be repeated here.

[0065] Through the present application, in the first controller, the first data set is sent to the data processing module, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; when the preset conditions are met, the data processing module splits the first data set to obtain multiple second data sets, and sends the multiple second data sets to the corresponding data storage, wherein the number of the second data sets is the same as the number of the data storage; the second data set is sent to the second controller through the transmission channel corresponding to the data storage. In the first controller, after obtaining the first data set, the first data set can be sent to the data processing module, so that the first data set is split in the data processing module to obtain multiple second data sets, and the second data set can be sent from the data processing module to the corresponding data storage. The second data set in the data storage can be sent to the second controller through the corresponding transmission channel. In order to realize the parallel sending of the first data set, the efficiency of data transmission is improved, thereby solving the technical problem of low data transmission efficiency between controllers in the current storage system in the related technology.

[0066] In an optional embodiment, when the preset conditions are met, the data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, including: in the data storage device, aggregating the multiple first data blocks included in the stored second data set to obtain aggregated second data blocks, and using the second data blocks to update the second data set to obtain an updated second data set.

[0067] In an optional implementation, the first data block may be an I / O request sent by the server, and the I / O request may be the amount of data read or written by the server. The size of the I / O request is arbitrary and depends on the requirements of the application and the design of the operating system. When writing data, the location of the data block on the storage device may be determined, and the logical block address may be converted to a physical block address, which may be accomplished by a block manager of the file system or firmware of the storage device.

[0068] It should be noted that an I / O request is an instruction issued by an operating system to read or write data to a storage device. In a block storage system, an I / O request is usually based on a data block.

[0069] Each data transmission needs to follow certain communication protocols, including establishing connections, transmission control, error detection and correction, etc. For small IO requests, these protocol overheads account for a larger proportion of the amount of data transmitted, because regardless of the size of the data packet, the protocol overhead is basically fixed. And each data packet needs to encapsulate header information, including source address, destination address, checksum, etc. For small I / O requests, the proportion of these header information relative to the data itself is high, resulting in a lower payload ratio for each data packet. In network transmission, each data packet requires a certain amount of time to process and transmit, including queuing and processing time in network devices. For small I / O requests, due to the small amount of data, the amount of data that can be transmitted per unit time is small, so more data packets are required to complete the transmission of the same amount of data. Therefore, small I / O requests require more interactions than large I / O requests, affecting the overall transmission efficiency.

[0070] Through the above-mentioned implementation of the present application, a second data set in a data storage device can be aggregated to obtain an aggregated second data block, and random small I / Os can be aggregated into sequential large I / Os to improve transmission performance in scenarios with high throughput and low latency requirements.

[0071] In an optional embodiment, the above-mentioned aggregating the multiple first data blocks included in the stored second data set includes one of the following: aggregating the multiple first data blocks included in the second data set stored in the data storage device into one second data block; aggregating the multiple first data blocks included in the second data set stored in the data storage device into multiple second data blocks, wherein the data size of the second data block is greater than the preset data size.

[0072] In an optional embodiment, the data processing module can be connected to a data storage device. The specific type of the data storage device is as described above. In one example, it can be a DDR. After the second data set is stored in the corresponding data storage device, the second data set can be aggregated in each data storage device to obtain a large I / O request. For the first data block to be merged, its size does not matter. Regardless of the size of the first data block, multiple first data blocks included in the second data set in a data storage device need to be aggregated, that is, there is no need to identify the size of the first data block during the aggregation process.

[0073] In an optional implementation, the first data blocks in a data storage device can be aggregated, and all the first data blocks in a second data set in a data storage device can be directly aggregated into a second data block as a whole without identifying the data blocks therein, so a large I / O request can be quickly obtained, and because the sizes of data contained in each second data set are similar, the sizes of second data blocks obtained in different data storage devices are similar, and sending the second data blocks in parallel can speed up data transmission efficiency.

[0074] In an optional implementation, the second data sets in the data storage device may be aggregated according to a preset data size, and different second data sets may be aggregated into multiple data blocks in the data storage device, and the data blocks have the same size, so as to improve transmission efficiency.

[0075] Through the above-mentioned implementation mode of the present application, the first data blocks therein can be directly aggregated through the data storage device, and the stored second data sets can be directly aggregated in each data storage device, and the sizes of the obtained second data blocks are close, so recalculation is avoided during the aggregation process, and the transmission efficiency is improved through large I / O requests during the sending process.

[0076] In an optional embodiment, the above-mentioned data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, including: sending the multiple second data sets obtained by splitting to the data storage device connected to the chip through the chip of the data processing module, wherein the number of chips of the data processing module is consistent with the number of data storage devices.

[0077] It should be noted that the data processing module can be a CPU, and the data processing module can be composed of a chip (Die). The chip is a small square cut from a wafer. It is the core part of the CPU and contains circuit logic for calculating, receiving / storing commands, and processing data.

[0078] It should be noted that the chip can be a large-area chip or multiple chips with specific functions obtained after being split, and multiple chips can be packaged into a data processing module. The chips can be interconnected through an off-chip bus, and each chip can be connected to a data storage device. The data stored in different data storage devices can be interoperable or stored separately.

[0079] Figure 3 is a schematic diagram of an optional data transmission method according to an embodiment of the present application; Figure 3As shown, the controller may include a data processing module and an IO card 302 and multiple data storage devices connected thereto. The data processing module may include multiple chips (such as chip one 304, chip two 306, chip three 308 and chip four 310), and the first data set is received from the server through the IO card 302, and then the first data set is split to obtain a second data set, which is sent to multiple chips respectively, and then sent to multiple connected data storage devices (storage one 312, storage two 314, storage three 316, storage four 318) through the chips. The second data set stored in multiple data storage devices can be aggregated to obtain a second data block. And sent to the second controller through the corresponding transmission channel.

[0080] Through the above implementation of the present application, there may be multiple chips in the data processing module, and each chip has a directly connected data storage device, and the multiple second data sets obtained by splitting can be sent to the multiple data storage devices respectively, so as to aggregate the data in the data storage devices to obtain the second data block. The multiple chips and the multiple data storage devices can be sent in parallel in the subsequent transmission process to improve the transmission efficiency.

[0081] In an optional implementation manner, when the preset conditions are met, the data processing module splits the first data set into multiple second data sets, including one of the following:

[0082] (1) when the sum of sizes of first data blocks included in the first data set is greater than a preset data threshold, splitting the first data set into multiple second data sets;

[0083] (2) Split a first data set consisting of first data blocks received within a first time into multiple second data sets.

[0084] It should be noted that after receiving data from the server, it is necessary to determine whether the data needs to be split. As mentioned above, splitting the data does not mean splitting a data block, but is equivalent to splitting a data set to obtain multiple data sets. Therefore, before splitting, it is necessary to determine the first data set to be split.

[0085] In an optional embodiment, the first data set can be determined by data volume or time. For example, when the first controller receives the first data block from the server, the size of the multiple first data blocks received can be calculated. When the total size of the multiple first data blocks exceeds the preset data threshold, the multiple first data blocks can be determined as the first data set. The preset data threshold can be a threshold preset according to actual needs. For example, in order to ensure the timeliness of the transmission, the preset data threshold can be set to a smaller value so that the first data set can be easily determined. In order to improve the utilization rate of hardware resources, the preset data threshold can be set to a larger value to increase the size of the second data block obtained subsequently, thereby increasing the amount of data transmitted each time. In one example, the first data set can also be determined according to time, and the first data block received within a certain period of time is determined as the first data set.

[0086] Through the above-mentioned implementation mode of the present application, the method for determining the first data set can be selected according to needs, the first data set can be determined according to time or the total size of data, and the preset data threshold and the value of the first time can be freely set according to actual needs to obtain the first data set that meets the project requirements, so as to facilitate subsequent splitting and transmission, thereby improving the reliability of data transmission.

[0087] In an optional embodiment, the data processing module splits the first data set into multiple second data sets, including: determining the total size of the first data blocks included in the first data set, and determining the mean data size based on the total size of the first data blocks and the number of data storage devices; adding the first data blocks included in the first data set in sequence as the i-th second data set until the total size of the first data blocks included in the i-th second data set is greater than the mean data size, thereby determining the i-th second data set; after determining the p-1-th second data set, determining the remaining first data blocks as the p-th second data set, where i is a positive integer greater than 0 and less than p, and p is the number of data storage devices.

[0088] It should be noted that, in the case where the first data set is determined by a preset data threshold, the mean of the preset data threshold may be calculated in advance and used as a basis for splitting to obtain the second data set.

[0089] It should be noted that although the first data set is split, the size of the second data set may be different. When the first data set is split according to the mean data size, the size of the first data block does not change. Therefore, for the first p-1 second data sets, its size is greater than or equal to the mean data size, and for the pth second data set, its size is smaller than the mean data size.

[0090] Figure 4is a schematic diagram of another optional data transmission method according to an embodiment of the present application; Figure 4 As shown, in the first controller, after receiving the first data set from the server, it can be equally divided and sent to the data storage device connected thereto through the chip in the data processing module to realize the storage of the second data set. The splitting process of the first data set can be performed in the data processing module, and the data processing module can also have a logic circuit dedicated to splitting or use any one of the multiple chips connected to the storage device to split the data.

[0091] Through the above implementation of the present application, different first data sets can be split according to the mean value to obtain approximately the same second data set, so that the size of data sent by each subsequent transmission channel is similar, thereby improving the transmission efficiency.

[0092] In an optional embodiment, the above-mentioned sending the second data set to the second controller through the transmission channel corresponding to the data storage device includes: sending the second data set to the bridge port; sending the second data set to the second controller through the bridge port and the transmission channel, wherein the transmission channel is connected to the data processing module in the second controller.

[0093] In an optional implementation, a physical PCIe Switch can be divided into multiple virtual partitions. The upstream port of each partition can be connected to a chip, and the downstream port can be connected to the data processing module of the second controller to form a transmission channel corresponding to each chip and the data storage device. The upstream port of each partition can be connected to the chip, that is, each partition can communicate with the data processing module without the need for other intermediate devices. Allowing each partition to have an independent data transmission path reduces data transmission delays and congestion. The independence between each partition means that they can run independently without interfering with each other. This design improves the flexibility and scalability of the system, and also enhances the fault tolerance of the system. If a partition fails, other partitions can still work normally.

[0094] Figure 5 is a schematic diagram of another optional data transmission method according to an embodiment of the present application; Figure 5 As shown, each chip may be connected to a transmission channel (such as transmission channel three 502, transmission channel one 504, transmission channel two 506, transmission channel four 508). The second data block in the data storage may be sent to the second controller through the corresponding transmission channel, thereby realizing parallel communication.

[0095] Through the above implementation of the present application, the aggregated second data blocks can be sent in parallel to the second controller through the corresponding transmission channels, which improves the efficiency of data transmission and the transmission bandwidth, thereby improving the efficiency of the storage system.

[0096] In an optional implementation, the sending of the second data set to the bridge port includes: when the second data set meets a sending condition, the memory access module sends the second data set in the data memory to the bridge port.

[0097] It should be noted that a DMA engine may exist in the PCIe Switch, and data can be directly transferred to the data storage device through the DMA engine without the intervention of the data processing module. The DMA engine can configure a specified channel to handle interrupts of completed, half-transmitted, and transmitted errors. After the transmission channel between the controllers is set, the source address (data source), destination address (data destination), and data block size to be transferred of the DMA controller can be set by the data processing module. During the operation of the storage system, after the first data set is split to obtain the second data set, the DMA controller can take over the control and perform the data transfer process. In this process, the data processing module does not need to perform data movement operations. The DMA controller continuously reads data from the corresponding data storage device and writes it to the corresponding address of the second controller until the second data set is transmitted. When the transmission is completed, the DMA controller will notify the data processing module through an interrupt signal that the transmission has ended, and the data processing module can handle other tasks or perform subsequent processing on the transmitted data.

[0098] Figure 6 is a schematic diagram of another optional data transmission method according to an embodiment of the present application; Figure 6 As shown, the DMA engine can detect whether there is data in the data memory, such as setting and identifying its data pointer, and actively moving the second data set when the pointer is not empty. Figure 6 As shown by the dotted lines, each data storage device may send the second data set to the second controller via a corresponding transmission channel.

[0099] Through the above-mentioned implementation mode of the present application, the second data set in the corresponding data storage device can be directly sent to the second controller through the transmission channel through the DMA engine without the intervention of the data processing module, so that the data can be moved quickly, the resources of the data processing module can be saved for other operations, and the overall performance of the storage system is improved.

[0100] In an optional implementation manner, the sending of the second data set to the bridge port includes: mapping a sending address of the second data set to obtain a receiving address corresponding to the second controller.

[0101] It should be noted that for the two controllers, they are two systems with independent address domains. Therefore, one side cannot directly observe the complete memory or I / O space of the other side. Therefore, there is a need for a port that can translate the two independent address domains, namely the non-transparent bridge (NTB). The data communication between the two controllers can be achieved through the special registers and address translation function of the non-transparent bridge.

[0102] In an optional implementation, after the storage system is determined, two sets of base address registers can be placed for the two controllers, one set for the first controller and the other set for the second controller. The base address registers can be used to define an address translation window to map the translated address to the data memory or I / O space, thereby achieving data transmission between the first controller and the second controller.

[0103] Through the above implementation of the present application, during the data transmission between two controllers, address translation can be implemented through a non-transparent bridge to avoid differences in address domains, so that the two controllers can communicate normally, thereby improving the accuracy of data transmission.

[0104] In an optional implementation, before sending the first data set to the data processing module, the step includes: acquiring the first data set at an input port.

[0105] In this embodiment, a data transmission system is provided. Figure 7 is an architecture diagram of an optional data transmission system according to an embodiment of the present application, such as Figure 7 As shown, the data transmission system includes: a first controller 708, wherein the first controller 708 includes a data processing module 704, a memory 706, a first PCIe switch 710, and an input port 702, and the first PCIe switch 710 is used to realize multi-channel data transmission with a second PCIe switch 714 of a second controller; a second controller 712, wherein the second controller 712 includes a second PCIe switch 714, and the second PCIe switch 714 is used to realize multi-channel data transmission with the first PCIe switch 710.

[0106] In an optional embodiment, the above-mentioned first PCIe switch 710 includes multiple partitions, and the uplink port of each partition of the first PCIe switch 710 is connected to the data processing module 704, wherein the uplink port is used for data transmission with the data processing module 704; the downlink port of each partition of the first PCIe switch 710 is connected to the downlink port of the corresponding partition of the second PCIe switch 714, wherein the downlink port is used to realize data transmission between the partition of the first PCIe switch 710 and the corresponding partition of the second PCIe switch 714.

[0107] In an optional embodiment, the uplink port of each partition of the above-mentioned first PCIe switch 710 is respectively connected to the corresponding chip in the data processing module 704, wherein the number of partitions of the first PCIe switch 710 is the same as the number of chips, and the partitions of the first PCIe switch 710 correspond one-to-one to the chips.

[0108] In an optional embodiment, each partition of the second PCIe switch 714 is respectively connected to the corresponding chip in the data processing module of the second controller 712, wherein the partition data of the second PCIe switch 714 is the same as the number of chips of the second controller 712, and the partitions of the second PCIe switch 714 correspond one-to-one to the chips of the second controller 712.

[0109] In an optional implementation, the chips of the data processing module 704 are interconnected via an off-chip bus, and the chips are respectively connected to corresponding memories 706 , wherein the memories 706 are used to store data blocks.

[0110] In an optional implementation, the first PCIe switch 710 includes a plurality of DMA engines, the number of the DMA engines being the same as the number of chips, and the DMA engines are used to retrieve data from the corresponding memory 706 .

[0111] In an optional embodiment, the first PCIe switch 710 and the second PCIe switch 714 are connected via a non-transparent bridge port, and the non-transparent bridge port is used to translate the data address in the first PCIe switch 710 into the address of the second PCIe switch 714 or translate the address of the second PCIe switch 714 into the address of the first PCIe switch 710.

[0112] In an optional implementation, the input port 702 is connected to a server and is used to receive data sent by the server.

[0113] Optionally, the first controller 708 and the second controller 712 can realize most of the advanced functions in the centralized storage system, such as disk management, abstracting disks into storage resource pools, and dividing logical unit numbers (LUNs) for clients to use. The first controller 708 and the second controller 712 are mutually active and standby to avoid hardware failures causing the entire storage system to be unavailable. The controller includes front-end ports and back-end ports. The front-end ports are connected to the outside to provide storage services, and the back-end ports are connected to more storage devices to form a larger storage resource pool and expand the capacity of the storage system.

[0114] Optionally, the data processing module 704 can be a CPU (Central Processing Unit), which is the computing core and control core of a computer. The CPU is composed of an arithmetic unit, a controller, and a register, as well as a bus that realizes the data, control, and status of the connections between them. The main function is to interpret computer instructions and process data in computer software. The CPU can be composed of multiple chips (CPU Die), each CPU Die contains multiple CPU cores. The CPU Dies are interconnected through an off-chip bus.

[0115] Optionally, the input port 702 may be an IO card, also known as an input / output card or an expansion card, which can be used to expand the hardware device of the server input / output interface. It mainly acts as a bridge between the server motherboard and external devices, responsible for transmitting data and control signals. IO cards are divided into front-end IO cards and back-end IO cards; front-end IO cards, such as FC cards and RoCE cards, are used to connect to servers; back-end IO cards, such as RoCE cards and SAS cards, are used to connect to hard disk expansion cabinets.

[0116] Optionally, the memory 706 can be a DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory). The data memory 706 can be triggered once at the rising edge and the falling edge of each clock cycle, respectively. This allows it to work twice in one clock cycle and complete twice the workload of ordinary memory in the same time.

[0117] Optionally, the first PCIe switch 710 and the second PCIe switch 714 may be composed of multiple PCIe ports and a switch core. When a PCIe device is connected to the PCIe switch, the switch connects the PCIe port of the device to the switch core. When multiple devices are connected to the switch, the switch forwards data from one port to another port as needed to achieve data transmission between multiple devices. In this way, the PCIe switch can connect multiple PCIe channels to a high-speed switch chip, thereby achieving more PCIe channels.

[0118] Optionally, a DMA engine may be provided in the first PCIe switch 710 and the second PCIe switch 714, and data may be quickly moved through DMA without CPU intervention, thereby saving CPU resources for other operations.

[0119] Optionally, a non-transparent bridge (NTB) can be used on the transmission channel. Since the two controllers belong to different address spaces, communication between the two parties can be achieved by performing address mapping and translation of corresponding data packets inside the PCIe Switch in two different partitions.

[0120] Through the above-mentioned implementation mode of the present application, the first data set transmitted from the server can be evenly distributed to the data storage device corresponding to the chip in the data processing module, and the aggregation of small IO requests can be realized in the data storage device to obtain large IO requests to improve the transmission rate. In the process of transmitting the second data block between the first controller and the second controller, each chip has an independent PCIe channel, which can ensure that it can be sent in parallel and improve the concurrent communication capability. In the process of sending the second data block to the second controller, inside the first controller, there is an independent DMA engine in the PCIe switch to carry data, without the participation of the data processing module, saving the resources of the data processing module. Improve the efficiency of data transmission, thereby solving the technical problem of low data transmission efficiency between controllers in the current storage system in the related art.

[0121] In this application, the front-end IO transmitted from the server is evenly distributed to each DIE of the CPU, and small IO and system messages are aggregated into large IO to improve transmission efficiency. The hardware cost does not increase. The PCIe switch wide port (X16) is split into narrow ports (X8), and the PCIe protocol is used to reduce the transmission delay across DIEs; the mirrored PCIe channel is distributed to each DIE to increase the concurrent transmission bandwidth. Increasing the concurrent transmission bandwidth and solving the single-channel performance bottleneck problem can achieve higher bandwidth.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the computer software product is stored in a storage medium (such as ROM / RAM, disk, CD), including several instructions for enabling a terminal device (which can be a computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0123] In this embodiment, a data transmission device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0124] Figure 8 is a structural block diagram of an optional data transmission device according to an embodiment of the present application, such as Figure 8 As shown, the device comprises:

[0125] A first sending module 82, configured to send a first data set to a data processing module in the first controller, wherein the first data set includes a plurality of first data blocks, and the data processing module is configured to process the data blocks;

[0126] A splitting module 84, configured to split the first data set into a plurality of second data sets by the data processing module when a preset condition is met, and send the plurality of second data sets to corresponding data storage devices respectively, wherein the number of the second data sets is the same as the number of the data storage devices;

[0127] The transmission module 86 is used to send the second data set to the second controller through the transmission channel corresponding to the data storage.

[0128] Through the present application, in the first controller, the first data set is sent to the data processing module, wherein the first data set includes multiple first data blocks, and the data processing module is used to process the data blocks; when the preset conditions are met, the data processing module splits the first data set to obtain multiple second data sets, and sends the multiple second data sets to the corresponding data storage, wherein the number of the second data sets is the same as the number of the data storage; the second data set is sent to the second controller through the transmission channel corresponding to the data storage. In the first controller, after obtaining the first data set, the first data set can be sent to the data processing module, so that the first data set is split in the data processing module to obtain multiple second data sets, and the second data set can be sent from the data processing module to the corresponding data storage. The second data set in the data storage can be sent to the second controller through the corresponding transmission channel. In order to realize the parallel sending of the first data set, the efficiency of data transmission is improved, thereby solving the technical problem of low data transmission efficiency between controllers in the current storage system in the related technology.

[0129] Optionally, the splitting module 84 is further used to: aggregate the multiple first data blocks included in the stored second data set in the data storage device to obtain aggregated second data blocks, and use the second data blocks to update the second data set to obtain an updated second data set.

[0130] Optionally, the above-mentioned aggregating the multiple first data blocks included in the stored second data set includes one of the following: aggregating the multiple first data blocks included in the second data set stored in the data storage device into one second data block; aggregating the multiple first data blocks included in the second data set stored in the data storage device into multiple second data blocks, wherein the data size of the second data block is greater than the preset data size.

[0131] Optionally, the splitting module 84 is further used to send the multiple second data sets obtained by splitting to a data storage device connected to the chip through the chip of the data processing module, wherein the number of chips of the data processing module is consistent with the number of data storage devices.

[0132] Optionally, the above-mentioned splitting module 84 is also used to: split the first data set into multiple second data sets when the total size of the first data blocks included in the first data set is greater than a preset data threshold; split the first data set composed of the first data blocks received within the first time into multiple second data sets.

[0133] Optionally, the splitting module 84 is further used to: determine the total size of the first data blocks included in the first data set, and determine a mean data size based on the total size of the first data blocks and the number of data storage devices; add the first data blocks included in the first data set in sequence as the i-th second data set until the total size of the first data blocks included in the i-th second data set is greater than the mean data size, and determine the i-th second data set; after determining the p-1-th second data set, determine the remaining first data blocks as the p-th second data set, where i is a positive integer greater than 0 and less than p, and p is the number of data storage devices.

[0134] Optionally, the sending of the second data set to the bridge port includes: when the second data set meets a sending condition, the memory access module sends the second data set in the data memory to the bridge port.

[0135] Optionally, the sending of the second data set to the bridge port includes: mapping a sending address of the second data set to obtain a receiving address corresponding to the second controller.

[0136] Optionally, the first sending module 82 is further configured to: obtain a first data set at an input port.

[0137] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0138] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0139] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0140] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0141] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0142] An embodiment of the present application further provides a computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program product, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0143] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0144] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0145] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data transmission method, characterized in that: The method comprises: In the first controller, a first data set is sent to a data processing module, wherein the first data set includes a plurality of first data blocks, and the data processing module is used to process the data blocks; The data processing module splits the first data set composed of the first data blocks received within the first time into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, wherein the number of the second data sets is the same as the number of the data storage devices; The second data set is sent to the second controller through the transmission channel corresponding to the data storage device, wherein the first controller is used to receive the first data set sent by the server, and the second controller is synchronized with the first controller data.

2. The method according to claim 1, characterized in that When a preset condition is met, the data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, including: In the data storage device, a plurality of the first data blocks included in the stored second data set are aggregated to obtain aggregated second data blocks, and the second data set is updated using the second data blocks to obtain an updated second data set.

3. The method according to claim 2, characterized in that The aggregating the plurality of first data blocks included in the stored second data set comprises one of the following: Aggregating a plurality of the first data blocks included in the second data set stored in the data storage device into one second data block; The plurality of first data blocks included in the second data set stored in the data storage device are aggregated into a plurality of second data blocks, wherein the data size of the second data block is greater than a preset data size.

4. The method according to claim 2, characterized in that: The data processing module splits the first data set into multiple second data sets, and sends the multiple second data sets to corresponding data storage devices respectively, including: The multiple second data sets obtained by the splitting are sent to a data storage device connected to the chip through the chip of the data processing module, wherein the number of chips of the data processing module is consistent with the number of the data storage devices.

5. The method according to claim 4, characterized in that When a preset condition is met, the data processing module splits the first data set into multiple second data sets, including: When the total size of the first data blocks included in the first data set is greater than a preset data threshold, the first data set is split into the multiple second data sets.

6. The method according to claim 4, characterized in that The data processing module splits the first data set into multiple second data sets, including: Determine a total size of the first data blocks included in the first data set, and determine a data size mean according to the total size of the first data blocks and the number of the data storage devices; Adding the first data blocks included in the first data set in sequence as the i-th second data set until the sum of the sizes of the first data blocks included in the i-th second data set is greater than the data size mean, and determining the i-th second data set; After the p-1th second data set is determined, the remaining first data blocks are determined as the pth second data set, where i is a positive integer greater than 0 and less than p, and p is the number of the data storage devices.

7. The method according to any one of claims 1 to 6, characterized in that: The sending the second data set to the second controller through the transmission channel corresponding to the data storage device includes: sending the second data set to a bridge port; The second data set is sent to the second controller through the bridge port and the transmission channel, wherein the transmission channel is connected to a data processing module in the second controller.

8. The method according to claim 7, characterized in that The sending the second data set to the bridge port comprises: In the case that the second data set meets the sending condition, the memory access module sends the second data set in the data memory to the bridge port.

9. The method according to claim 7, characterized in that: The sending the second data set to the bridge port comprises: The sending address of the second data set is mapped to obtain the receiving address corresponding to the second controller.

10. The method according to claim 1, characterized in that Before sending the first data set to the data processing module, the method includes: The first data set is obtained at an input port.

11. A data transmission system, characterized in that: The system comprises: A first controller, wherein the first controller comprises a data processing module, a data storage device, a first PCIe switch, and an input port, and the first PCIe switch is used to realize multi-channel data transmission with a second PCIe switch of the second controller; A second controller, wherein the second controller comprises a second PCIe switch, and the second PCIe switch is used to realize multi-channel data transmission with the first PCIe switch.

12. The system according to claim 11, characterized in that The first PCIe switch includes a plurality of partitions, and an uplink port of each partition of the first PCIe switch is connected to the data processing module, wherein the uplink port is used for data transmission with the data processing module; The downlink port of each partition of the first PCIe switch is connected to the downlink port of the corresponding partition of the second PCIe switch, wherein the downlink port is used to implement data transmission between the partition of the first PCIe switch and the corresponding partition of the second PCIe switch.

13. The system according to claim 12, characterized in that The uplink port of each partition of the first PCIe switch is respectively connected to the corresponding chip in the data processing module, wherein the number of partitions of the first PCIe switch is the same as the number of chips, and the partitions of the first PCIe switch correspond one-to-one to the chips.

14. The system according to claim 13, characterized in that Each partition of the second PCIe switch is respectively connected to the corresponding chip in the data processing module of the second controller, wherein the partition data of the second PCIe switch is the same as the number of chips of the second controller, and the partitions of the second PCIe switch correspond one-to-one to the chips of the second controller.

15. The system according to claim 13, characterized in that The chips of the data processing module are interconnected via an off-chip bus, and the chips are respectively connected to corresponding data storage devices, wherein the data storage devices are used to store data blocks.

16. The system according to claim 13, characterized in that The first PCIe switch includes a plurality of DMA engines, the number of the DMA engines is the same as the number of the chips, and the DMA engines are used to fetch data from corresponding data storage devices.

17. The system according to claim 16, characterized in that The first PCIe switch and the second PCIe switch are connected via a non-transparent bridge port, and the non-transparent bridge port is used to translate the data address in the first PCIe switch into the address of the second PCIe switch or translate the address of the second PCIe switch into the address of the first PCIe switch.

18. The system according to any one of claims 11 to 17, characterized in that The input port is connected to the server and is used to receive data sent by the server.

19. A data transmission device, characterized in that: The device comprises: A first sending module, configured to send a first data set to a data processing module in a first controller, wherein the first data set includes a plurality of first data blocks, and the data processing module is configured to process the data blocks; a splitting module, used for the data processing module to split the first data set composed of the first data blocks received within the first time into multiple second data sets, and send the multiple second data sets to corresponding data storage devices respectively, wherein the number of the second data sets is the same as the number of the data storage devices; The transmission module is used to send the second data set to the second controller through the transmission channel corresponding to the data storage device, wherein the first controller is used to receive the first data set sent by the server, and the second controller is synchronized with the first controller data.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method according to any one of claims 1 to 10 when executed by a processor.

21. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

22. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images