Method for storing data, device, storage medium, and program product

Abstract

The present disclosure belongs to the technical field of computers, and provides a method for storing data, a device, a storage medium, and a program product. In the method, a network card divides received target data to be stored and distributes the divided data to a plurality of processor sets, and the plurality of processor sets each perform data storage-related processing. Compared to a network card sending the entire volume of data to a single processor set and then the processor set distributing same to other processor sets, in the present solution, the network card directly distributes the data to the processor sets, and the received data does not need to be distributed to other processor sets by a single processor set. In this way, data transmission between processor sets can be reduced, and the processor sets are connected by means of an inter-chip bus, so that the occupation of the inter-chip bus is reduced.

Description

Data storage method, device, storage medium and program product

[0001] This disclosure claims priority to Chinese patent application number 202410232175.9, filed on February 29, 2024, entitled “Data Storage Method, Device, Storage Medium and Program Product,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to a data storage method, device, storage medium, and program product. Background Art

[0003] With the development of computer technology, storage devices have become widely used. High-performance storage devices are generally equipped with multiple processors. When a network interface card receives data that needs to be stored, it sends the data to one processor. This processor then divides and distributes the data to other processors, with each processor responsible for storage-related processing of the received data. However, this method results in excessive data forwarding by the processors, significantly consuming the bandwidth of the inter-chip bus between the processors. Summary of the Invention

[0004] The present disclosure provides a data storage method capable of reducing the bandwidth occupied by the processor inter-chip bus. The technical solution is as follows:

[0005] In a first aspect, a data storage method is provided, which is applied to a target device. The target device includes a network interface card (NIC) and multiple processor assemblies, where the processor assemblies may consist of a single processor or multiple processors. The method includes: the NIC receiving target data to be stored, dividing the target data into multiple sub-data, and distributing the data to each processor assembly; each processor assembly then generating corresponding redundant data based on the received sub-data, and storing the received sub-data and the corresponding redundant data.

[0006] The processor set that distributes data via the network card can be part of the target device's processor set or all of its processor sets. The network card can distribute multiple sub-data directly to each processor set without requiring other processor sets to relay the data.

[0007] In the process of continuously receiving target data, the network card chip can store the received target data in the cache and perform statistics on the received target data. When certain conditions are met, a processor set can be selected from multiple processor sets according to the specified load balancing mechanism, and the cached target data can be sent to the processor set, and the cache can be cleared. The target data in the cache sent at this time can be considered as a piece of sub-data. There are many possibilities for the above conditions, for example, the amount of target data to be distributed in the cache reaches a specified data amount threshold, the number of cached data packets reaches a specified number threshold, reaches a specified time period, and so on. There are also many possibilities for the above load balancing mechanism, which can be a polling principle, or it can be to select the processor set that occupies the least resources each time, or it can be a back pressure mechanism, and so on.

[0008] In this way, the network card divides the received target data to be stored and distributes it to each processor set, and each processor set performs data storage-related processing separately. Compared with the network card sending the entire amount of data to a processor set and then distributing it to other processor sets by the processor set, this solution is that the network card distributes data directly to the processor set, without the need for one processor set to distribute the data to other processor sets. Moreover, the data distributed by the network card to any processor set does not need to be transferred through other processor sets. In this way, data transmission between processor sets can be reduced. The processor sets are connected through the inter-chip bus, so this solution can reduce the bandwidth occupied by the inter-chip bus.

[0009] In a possible implementation, the network card communicates with each processor set through different buses, and the network card sends the sub-data to the corresponding processor set through the bus corresponding to each processor set.

[0010] In this way, the data distributed by the network card to any processor set does not need to be transferred through other processor sets. The data can be distributed directly to any processor set without being transferred through other processor sets, which can reduce data transmission between processor sets.

[0011] In a possible implementation, the target device includes multiple mainboards, and at least two processor sets among the multiple processor sets are installed on each mainboard.

[0012] In this way, when a problem occurs on a motherboard or a processor set on a motherboard, data can be stored or read through another motherboard and the corresponding processor set, thereby ensuring normal data reading and writing and improving data security.

[0013] In a possible implementation, the number of processor sets located on each mainboard in the multiple processor sets is the same.

[0014] This way, the control logic of the processor set is simpler.

[0015] In a possible implementation, the network card divides the received target data into multiple sub-data based on a load balancing mechanism, and distributes the multiple sub-data to each processor set.

[0016] This allows each processor set to achieve load balancing, improve data storage speed, and extend the service life of each device component.

[0017] In one possible implementation, the target device includes multiple memories, and different processor sets correspond to different memories. Each processor set sends received sub-data to the corresponding memory for storage, and sends redundant data to at least one other processor set. Each other processor set then sends the received redundant data to the corresponding memory for storage.

[0018] In this way, the sub-data and the corresponding redundant data are stored in memories managed by different processor sets, which greatly reduces the probability of simultaneous loss of sub-data and redundant data. As long as the two are not lost at the same time, data recovery can be performed, thereby improving data security.

[0019] In one possible implementation, each processor set sends redundant data to at least one other processor set that belongs to the same motherboard as the processor set.

[0020] This way, if sub-data is lost, data can be recovered using redundant data from other processor sets on the same motherboard. Data transfer between processor sets on the same motherboard is faster than between processor sets on different motherboards, thus increasing the speed of data recovery.

[0021] In one possible implementation, the network card copies each sub-data to obtain corresponding mirror data, and distributes the copied mirror data to each processor set. The mirror data and corresponding sub-data are sent to different processor sets. The processor set caches the received mirror data. When the sub-data corresponding to the cached mirror data is successfully stored, the cached mirror data is deleted. When the sub-data corresponding to the cached mirror data fails to be stored, the cached mirror data is stored in the memory.

[0022] While continuously receiving data, the NIC chip can store the received data in a cache. When certain conditions are met, it can select a processor set (referred to as the first processor set) from among multiple processor sets and send the cached data to that processor set. At this point, the NIC chip can copy the data in the cache to obtain mirrored data. The NIC chip then selects another processor set (referred to as the second processor set) to send the mirrored data to and sends the mirrored data to the second processor set.

[0023] The network card chip can select the second processor set in the following ways: Method 1: Pre-establish a correspondence between processor sets, in which two processor sets correspond to each other, and the two corresponding processor sets are connected to the same memory. Method 2: Randomly select the second processor set from processor sets other than the first processor set. Method 2 is applicable when multiple processor sets are connected to all memories.

[0024] In this way, when data fails to be written into the memory, the mirror data can be written into the memory to ensure that the data can be stored in the memory, thereby improving data security.

[0025] In a possible implementation, the mirror data and the corresponding sub-data are respectively sent to processor sets on different mainboards.

[0026] In this way, when a mainboard of a sub-data fails and the data cannot be stored in the memory, the mirror data on other mainboards can be used to ensure that the data can be stored in the memory, thereby improving data security.

[0027] In a second aspect, a computer device is provided, which includes a network card and a plurality of processor sets, and is used to execute the method provided by the first aspect and its possible implementation methods.

[0028] In a third aspect, a computer-readable storage medium is provided, which stores computer program code. When the computer program code is executed by a computer device, the computer device executes the method provided by the first aspect and its possible implementation methods.

[0029] In a fourth aspect, a computer program product is provided. The computer program product includes computer program code. When the computer program code is executed by a computer device, the computer device executes the method provided by the first aspect and its possible implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is a schematic structural diagram of a target device provided by an embodiment of the present disclosure;

[0031] FIG2 is a schematic structural diagram of a target device provided by an embodiment of the present disclosure;

[0032] FIG3 is a schematic structural diagram of a target device provided by an embodiment of the present disclosure;

[0033] FIG4 is a schematic structural diagram of a target device provided by an embodiment of the present disclosure;

[0034] FIG5 is a schematic diagram of a data storage processing method according to an embodiment of the present disclosure;

[0035] FIG6 is a schematic diagram of a process flow for generating and distributing mirror data provided by an embodiment of the present disclosure;

[0036] FIG7 is a schematic diagram of a data storage processing flow provided by an embodiment of the present disclosure;

[0037] FIG8 is a schematic diagram of data processing and data transmission in a data storage process provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The following is a brief description of the terms involved in the embodiments of the present disclosure.

[0039] Peripheral component interconnect express (PCIE):

[0040] PCIE is a high-speed serial computer expansion bus standard. PCIE is commonly used for communication between components within a device, primarily between the CPU and other components. Devices can be equipped with PCIE interfaces to connect various PCIE-enabled components, such as hard drives, network cards, graphics cards, and compute cards. PCIE interfaces can be provided on the motherboard or backplane and connect to the CPU, which has an integrated PCIE controller.

[0041] Remote direct memory access (RDMA):

[0042] RDMA is a technology that allows external devices to directly access data stored in local device storage (such as RAM and hard disks), bypassing the central processing unit (CPU). RDMA is commonly used for communication between devices or between boards within a device. Inter-board communication can be between different motherboards or between a motherboard and a backplane.

[0043] The present disclosure provides a data storage method. This method is applied to a target device, which can be a computer device with storage and certain processing capabilities. The present disclosure uses a storage device as an example to illustrate the method. Other scenarios are similar and are not described in detail. A storage device is a computer device used for data storage. For example, a storage device can be a device in a storage system cluster that ultimately stores data, or a device with an independent and complete storage system built within it.

[0044] The target device may include multiple processor sets and network cards. Each processor set may include one or more processors. The disclosed embodiment uses an example in which each processor set includes one processor to provide a detailed description of the solution. Other cases are similar and will not be described in detail. The structure of the target device may be as shown in FIG1 , including multiple processors 110 and a network card 120.

[0045] The processor 110 can be any one or more of a CPU, a graphics processing unit (GPU), a data processing unit (DPU), a microprocessor (MP), or a digital signal processor (DSP). The processor 110 can be used to process data involved in the method, for example, to generate redundant data. The embodiment of the present disclosure takes the CPU as an example to explain the solution in detail. Other situations are similar and will not be described in detail. Various types of controllers can be integrated into the CPU, such as a network adapter, a PCIE controller, an inter-chip bus controller, etc.

[0046] The network card 120 can communicate with multiple processors 110 via different buses. The network card 120 can be used to facilitate communication between the computer device in which it is located and other devices or communication networks. It can also be used to facilitate communication between different motherboards within the computer device in which it is located. The network card 120 can also communicate with the processors via buses. For example, the network card can receive data from external devices and send it internally to the CPU and other components. It can also send data obtained from the CPU or memory to external devices. Furthermore, the network card 120 has the ability to replicate data.

[0047] The target device may also include a memory 130, which may be a large-capacity storage medium such as a solid state drive (SSD) or a mechanical hard disk. The memory 130 may be connected to a backplane or a motherboard. Different types of memory 130 have different types of interfaces, and the memory may communicate with the processor 110 or the network card 120 through different interfaces. The memory 130 may be used for persistent storage of data, for example, storing the original data and corresponding redundant data of a database. In another possible solution, the target device may not include the memory 130, and the target device may use an external memory for data storage.

[0048] The internal connection framework of the target device can be constructed using the mainboard 140, and can include one or more mainboards 140. The internal connection framework of the target device can also be constructed using both the mainboard 140 and the backplane 150, and can include the backplane 150 and multiple mainboards 140, with the multiple mainboards 140 connected to the backplane 150. The following describes several specific scenarios.

[0049] Case 1, the target device has a motherboard 140

[0050] 1.1 Motherboard 140

[0051] The motherboard 140 may be provided with multiple processor interfaces, and may also be provided with interfaces such as PCIE interfaces, SATA interfaces, and Ethernet interfaces. The processor interface is generally used to connect to the processor, the PCIE interface can be used to connect to the network card 120, the memory 130, etc., the SATA interface can be used to connect to the memory 130, and the Ethernet interface can be used to connect to other devices. There are many alternative interfaces for connecting to the network card 120 and the memory 130. The embodiment of this disclosure mainly uses the PCIE interface as an example to explain the solution in detail. Other situations are similar and will not be repeated here.

[0052] 1.2 Processor 110

[0053] Each processor 110 may be mounted on a processor interface of the motherboard 140. The processor interfaces may be connected via an inter-chip bus to enable communication between different processors 110. An inter-chip bus controller may be integrated in the processor 110.

[0054] 1.3 Network Card 120

[0055] The network card 120 can be installed on the PCIe interface of the motherboard 140. The PCIe interface can be connected to the processor interface via a bus to enable communication between the network card 120 and the processor 110. A PCIe controller can be integrated into the processor 110. The number of network cards 120 connected to the motherboard 140 can be set based on the device's external communication bandwidth or port number requirements, such as 1, 2, or 3.

[0056] 1.4 Memory 130

[0057] The memory 130 can also be installed on the PCIE interface (or SATA interface) of the motherboard 140. The bus connection between the PCIE interface and the processor interface enables communication between the memory 130 and the processor 110. The number of memories 130 connected to the motherboard 140 can be set based on the device's storage capacity and storage security requirements, such as 1, 2, 3, etc. If there are multiple memories 130, the multiple memories 130 can be assigned to different processors 110 for control. For example, if there are 4 processors 110 and 36 memories 130, each processor 110 can control 9 memories 130. Each processor 110 is directly connected to one or more memories 130, and each processor 110 is responsible for controlling the directly connected memories 130. This facilitates simultaneous load balancing of multiple memories 130 when load balancing the processors 110. In this process, data transmission between different processors 110 can be reduced, thereby reducing the occupancy of the inter-chip bus.

[0058] The following are several possible structures of the target device in case 1:

[0059] Structure 1, as shown in Figure 2, involves a target device consisting of a motherboard 140, two processors 110 (CPU1 and CPU2 in the figure), a network card 120, and two memories 130 (SSD1 and SSD2 in the figure, or alternatively, two sets of memories 130). The network card is connected to CPU1 and CPU2 via P0 (Pi represents a PCIE interface, where i = 1, 2, ...). CPU1 and CPU2 are connected via an inter-chip bus. SSD1 is connected to CPU1 via P1, and SSD2 is connected to CPU2 via P2.

[0060] Structure 2, as shown in Figure 3, the target device includes a motherboard 140, four processors 110 (CPU1, CPU2, CPU3, CPU4 in the figure), four network cards 120 (only one of the network cards is connected to each CPU in the figure, and the connections between the other three network cards and the CPU are similar and omitted in the figure) and four memories 130 (SSD1, SSD2, SSD3, SSD4 in the figure, which can also be four groups of memories 130). The network card is connected to CPU1, CPU2, CPU3, and CPU4 through P0 (Pi represents the PCIE interface, i=1, 2, ...). Any two CPUs are connected through an inter-chip bus. SSD1 is connected to CPU1 through P1, SSD2 is connected to CPU2 through P2, SSD3 is connected to CPU3 through P3, and SSD4 is connected to CPU4 through P4.

[0061] Case 2: The target device has multiple motherboards 140, which are connected to each other via a backplane 150.

[0062] 2.1 Motherboard 140

[0063] Each motherboard 140 may be provided with multiple processor interfaces, and may also be provided with interfaces such as PCIE interfaces, SATA interfaces, and Ethernet interfaces. Processor interfaces are generally used to connect to processors, PCIE interfaces can be used to connect to network cards 120, memory 130, etc., SATA interfaces can be used to connect to memory 130, and Ethernet interfaces can be used to connect to other devices. PCIE interfaces, SATA interfaces, and Ethernet interfaces can also be used for inter-board connections. There are many alternative interfaces for connecting network cards 120, memory 130, and inter-board connections. The embodiments of this disclosure mainly use the PCIE interface as an example to explain the detailed solution. Other situations are similar and will not be described in detail.

[0064] 2.2 Backplane 150

[0065] The backplane 150 may be provided with multiple interfaces, such as PCIE interfaces, SATA interfaces, and Ethernet interfaces. The backplane 150 can be connected to the PCIE interfaces of the motherboard 140 via the PCIE interfaces. Various components, such as the memory 130, can be installed on the backplane 150 via the PCIE interfaces. Furthermore, the backplane 150 may be provided with a direct line to directly connect two PCIE interfaces. These two PCIE interfaces can be used to connect a component to a motherboard 140, thereby enabling a connection between a component on the backplane 150 and the motherboard 140; alternatively, these two PCIE interfaces can be connected to PCIE interfaces on two different motherboards 140, thereby enabling a connection between two motherboards 140.

[0066] 2.3 Processor 110

[0067] Each processor 110 can be installed on the processor interface of the motherboard 140. The processor interfaces on the same motherboard can be connected through an inter-chip bus to realize communication between different processors 110 on the same motherboard, and an inter-chip bus controller can be integrated in the processor 110. The processors 110 on different motherboards communicate with each other through the PCIE interface (or Ethernet interface) of the motherboard and the backplane, and a PCIE controller or network adapter can be integrated in the processor 110 accordingly. Optionally, the processor 110 can communicate with other processors on different motherboards 140 through the backplane 150 (each motherboard 140 and the backplane 150 are connected through a PCIE interface or an Ethernet interface), or, the backplane 150 can be skipped, and the processor 110 can communicate with the processors 110 on other motherboards 140 through the connection between the motherboards 140 (different motherboards 140 are connected through a PCIE interface or an Ethernet interface).

[0068] 2.4 network card 120

[0069] The network card 120 can be installed on a PCIE interface of the motherboard 140. The PCIE interface can be connected to the processor interface via a bus to enable communication between the network card 120 and the processor 110 on the same motherboard. Accordingly, a PCIE controller can be integrated into the processor 110. The network card 120 can communicate with the processors 110 on other motherboards 140 via the PCIE interface (or Ethernet interface) of the motherboard 140 and the backplane 150. Accordingly, a PCIE controller or network adapter can be integrated into the processor 110. Optionally, the network card 120 can communicate with processors 110 on different motherboards 140 via the backplane 150 (each motherboard 140 is connected to the backplane 150 via a PCIE interface or Ethernet interface). Alternatively, the backplane 150 can be skipped, and the network card 120 can communicate with the processors 110 on other motherboards 140 via connections between motherboards 140 (different motherboards 140 are connected via PCIE interfaces or Ethernet interfaces). The number of network cards 120 connected to the mainboard 140 can be set based on the device's external communication bandwidth or port number requirements, such as 1, 2, 3, etc.

[0070] Optionally, if the target device has a backplane 150, the network card 120 may also be installed on a PCIE interface of the backplane 150. The connection between the network card 120 and the processor 110 may be achieved through the connection between the mainboard 140 and the backplane 150.

[0071] 2.5 Memory 130

[0072] The memory 130 can be installed on the PCIE interface (or SATA interface) of the backplane 150. The PCIE interface connects to the processor interface on the motherboard 140 through the connection between the backplane 150 and the motherboard 140, enabling communication between the memory 130 and the processor 110. The number of memories 130 connected to the backplane 150 can be set based on the device's storage capacity and storage security requirements, such as one, two, or three. If there are multiple memories 130, they can be assigned to different processors 110 for control. Accordingly, through the connection between the motherboard 140 and the backplane 150, each processor 110 is connected to one or more memories 130 on the backplane 150, and each processor 110 is configured to control the memory 130 to which it is connected. Any memory 130 can be connected to at least one processor on each motherboard 140. In this way, if one motherboard 140 fails, the processor 110 on another motherboard 140 can read and write data to the corresponding memory 130.

[0073] In addition, optionally, the memory 130 may also be installed on the PCIE interface (or SATA interface) of the mainboard 140 .

[0074] The following is a possible structure of the target device in case 2:

[0075] As shown in Figure 4, the target device includes two motherboards 140, a backplane 150, four processors 110 (CPU1, CPU2, CPU3, and CPU4 in the figure), four network cards 120 (only one of the network cards is shown connected to each CPU; the connections between the other three network cards and the CPU are similar and omitted in the figure), and four memories 130 (SSD1, SSD2, SSD3, and SSD4 in the figure, which can also be four groups of memories 130). Network card 120 is connected to CPU1 and CPU2 via P0 (Pi represents a PCIE interface, i = 1, 2, ...). The connection between network card 120 and CPU3 is: network card 120 → P1 → P2 → P3 → P4 → CPU3. The connection between network card 120 and CPU4 is: network card 120 → P5 → P6 → P7 → P8 → CPU4. CPU1 and CPU2 are on the same motherboard 140 and are connected by an inter-chip bus. CPU3 and CPU4 are on another motherboard 140 and are connected by an inter-chip bus. The connection between CPU2 and CPU3 is: CPU2 → P9 → P10 → P11 → P12 → CPU3. The connection between CPU1 and CPU4 is: CPU1 → P13 → P14 → P15 → P16 → CPU4. The four memories 130 are connected to the backplane 150. The connection between CPU1 and SSD1 is: CPU1 → P17 → P18 → P19 → SSD1. The connections between other CPUs and SSDs are shown in the attached figure and are not listed here.

[0076] The present disclosure provides a data storage method, which is applied to a target device. The corresponding processing flow may be shown in FIG5 , including the following steps:

[0077] 501. The network card receives target data to be stored.

[0078] The target device's network interface card (NIC) is equipped with an interface for connecting to external devices (this disclosure uses an Ethernet interface as an example; other similar cases are not described here). It also includes a NIC chip and a cache. The Ethernet interface receives data sent to the target device by other devices, which is data that the target device needs to store. The Ethernet interface transmits the received data to the NIC chip.

[0079] 502. The network card divides the received target data into multiple sub-data, and distributes the multiple sub-data to each processor.

[0080] The multiple processors to which the network card distributes data can be some or all of the processors on the target device. For example, if the target device has two motherboards, each with two CPUs installed, each network card can distribute data to all four CPUs. Alternatively, the network card can distribute data to both CPUs on its motherboard, and so on.

[0081] As the network card chip continuously receives data, it can store the received data in a cache and collect statistics on it. When certain conditions are met, it can select a processor from among multiple processors according to a specified load balancing mechanism and send the cached data directly to that processor. The network card is connected to each processor via a bus, and the network card sends data to the corresponding processor via the corresponding bus without transiting through other processors. The network card chip can then clear the cache. The data in the cache sent at this time can be considered as a sub-data.

[0082] There are many possibilities for the above conditions: the amount of data to be distributed in the cache may reach a specified data amount threshold (the data amount threshold may be pre-set by technical personnel based on experience, for example, the amount of data required to generate redundant data may be considered), for example, the data amount threshold may be 1 kilobyte (KB), 16KB, etc.; the condition may also be that the number of cached data packets reaches a specified number threshold (the number threshold may also be pre-set by technical personnel based on experience, for example, the general size of received data packets and the amount of data required to generate redundant data may be considered), for example, the number threshold of data packets may be 2, 3, etc.; the condition may also be that a specified time period is reached (the time period may also be pre-set by technical personnel based on experience, for example, the data reception rate and the amount of data required to generate redundant data may be considered), for example, the time period is 1 second, 2 seconds, etc.

[0083] The network card can also select different conditions according to the service type. A correspondence table between service types and conditions can be established in advance. Based on the service type of the currently received data, the corresponding condition of the service type can be found in the correspondence table, and the above distribution processing can be performed based on the found condition.

[0084] There are also many possibilities for the above-mentioned load balancing mechanism. One of them is to select a processor from multiple processors using the polling principle. For example, each time the network card distributes data, it can select a processor from the four processors CPU1, CPU2, CPU3, and CPU4. The network card can select processors in the order of CPU1→CPU2→CPU3→CPU4→CPU1→... The load balancing mechanism can also be that the network card selects the processor with the least resource occupancy based on the resource occupancy information of multiple processors. The load balancing mechanism can also be a back pressure mechanism, that is, the network card chip can continuously send data to a processor. When the processor detects that its own resource occupancy rate reaches a threshold, it sends a decompression notification to the network card. After receiving the notification, the network card chip can stop sending data to the processor and start sending data to the next processor, and so on.

[0085] 503. Each processor generates corresponding redundant data based on the received sub-data.

[0086] To enable data recovery when partial data is lost, each processor can perform redundancy processing based on the received sub-data to generate redundant data. Redundancy processing can be performed in a variety of ways, such as erasure coding (EC) calculation. The disclosed embodiment uses EC calculation as an example to explain in detail. The redundant data generated by EC calculation can generally be referred to as verification data. Other situations are similar and will not be described in detail. Technicians can pre-set the redundancy ratio of EC calculation. The redundancy ratio is the ratio of the amount of verification data to the total amount of data, where the total amount of data is the sum of the data for EC calculation and the amount of verification data obtained by calculation.

[0087] During EC calculations, the sub-data can be divided into multiple data blocks, each of which contains a fixed amount of data. A specified number of data blocks (the specified number can be pre-set based on experience) are grouped together, and an EC calculation is performed on each group of data blocks to obtain the corresponding check data. For example, when the redundancy ratio is 50% and the specified number is 3, EC calculations are performed on three data blocks each time to obtain three check data blocks. When the redundancy ratio is 40% and the specified number is 3, EC calculations are performed on three data blocks each time to obtain two check data blocks.

[0088] At 504 , each processor stores the received sub-data and corresponding redundant data in a memory.

[0089] Typically, a device will have multiple memories. A pre-established relationship between processors and memories can be established, where different processors correspond to different memories, and each processor is responsible for reading and writing data to the corresponding memory. This relationship can be established based on a direct connection between the processor and the memory.

[0090] Each processor can be directly connected to at least one memory. The so-called direct connection between the processor and the memory means that there is no other processor on the connection line between the processor and the memory (of course, there may be some devices for signal enhancement on the connection line, which does not affect the direct connection relationship between them). For example, if a PCIE interface is directly connected to a processor interface, then the memory plugged into the PCIE interface is directly connected to the processor plugged into the processor interface. For another example, PCIE interface 1 on the motherboard is directly connected to PCIE interface 2 on the backplane. On the motherboard, PCIE interface 1 is directly connected to processor interface 1, and on the backplane, PCIE interface 2 is directly connected to PCIE interface 3. Then, the memory plugged into PCIE interface 3 is directly connected to the processor plugged into processor interface 1.

[0091] When each memory is directly connected to a processor (called connection mode 1), the processor is responsible for reading and writing data to the directly connected memory. When each memory is directly connected to multiple processors (called connection mode 2), one of the multiple processors directly connected to the memory is the main processor of the memory, and the other processors are backup processors of the memory. Under normal circumstances, the main processor is responsible for reading and writing data to the memory. Under abnormal circumstances (such as a main processor failure or a failure in the connection between the main processor and the memory), the backup processor is responsible for reading and writing data to the memory. For example, as shown in Figure 4, the processors directly connected to SSD1 include CPU1 and CPU3, where the main processor is CPU1 and the backup processor is CPU3. The processors directly connected to SSD2 include CPU2 and CPU4, where the main processor is CPU2 and the backup processor is CPU4. The processors directly connected to SSD3 include CPU1 and CPU3, where the main processor is CPU3 and the backup processor is CPU1. The processors directly connected to SSD4 include CPU2 and CPU4, where the main processor is CPU4 and the backup processor is CPU2.

[0092] For the above-mentioned connection mode 1, the memory directly connected to the processor can be set as the memory corresponding to the processor to obtain the corresponding relationship between the processor and the memory.

[0093] For the second connection method described above, the main processor directly connected to the memory can be set as the processor corresponding to the memory to obtain a corresponding relationship between the processor and the memory. Optionally, different processors in this corresponding relationship will not correspond to the same memory. In this way, distributing data to the processor is equivalent to allocating memory for the data. For example, as shown in Figure 4, the processors directly connected to SSD1 include CPU1 and CPU3, where CPU1 is the main processor of SSD1 and CPU3 is the backup processor of SSD1. The main processor CPU1 can be set as the processor corresponding to SSD1 and added to the corresponding relationship between the processor and the memory.

[0094] In the case of the second connection mode, the processor may send the sub-data and redundant data to the memory in the following manner:

[0095] In the first processing method, after performing redundancy processing, a processor sends the sub-data to the corresponding memory for storage and sends the redundant data to at least one other processor. Each other processor then sends the redundant data to its corresponding memory for storage. This way, the sub-data and redundant data are stored in different memories, reducing the possibility of simultaneous loss of both sub-data and redundant data, thereby effectively improving data security.

[0096] Optionally, when sending redundant data, a processor can select other processors on the same motherboard and send the redundant data to these other processors. In this way, when performing data recovery, data transmission between processors on the same motherboard is faster than data transmission between processors on different motherboards, thereby improving the speed of data recovery.

[0097] For example, as shown in FIG4 , after CPU1 performs redundancy processing on the received sub-data, it sends the sub-data to SSD1 for storage and sends the redundant data to CPU2 , and CPU2 sends the redundant data sent by CPU1 to SSD2 for storage.

[0098] Processing method 2: After performing redundancy processing, the processor sends the sub-data to a memory directly connected to the processor for storage, and sends the redundant data to another memory directly connected to the processor for storage.

[0099] For example, as shown in FIG4 , after performing redundancy processing on the received sub-data, CPU1 sends the sub-data to SSD1 for storage, and sends the redundant data to SSD3 for storage.

[0100] In the embodiment of the present disclosure, mirror data may be generated for the data to be stored. When data fails to be written to the memory, the mirror data may be written to the memory. The corresponding processing flow may be shown in FIG6 , including the following steps:

[0101] 601. The network card copies each sub-data to obtain corresponding mirror data.

[0102] As described in the preceding process, the NIC chip can store the received data in a cache while continuously receiving it. When certain conditions are met (the various possibilities of these conditions have been detailed above), it selects a processor (referred to as the first processor) from among the multiple processors and sends the cached data to it. At this point, the NIC chip can copy the data in the cache to obtain mirrored data.

[0103] For example, when the network card is receiving data, the sub-data sent to each processor in sequence are A1, A2, A3, A4, ..., and these sub-data are sent to CPU1, CPU2, CPU3, CPU4, ... respectively according to the polling principle. When the network card determines A1, it generates mirror data A1' of A1; when it determines A2, it generates mirror data A2' of A2; when it determines A3, it generates mirror data A3' of A3; and when it determines A4, it generates mirror data A4' of A4.

[0104] 602. The network card distributes the copied mirror data to each processor.

[0105] 603. The processor caches the received mirror data.

[0106] 604 , when the sub-data corresponding to the cached mirror data is successfully stored, the processor deletes the cached mirror data.

[0107] 605 , when the storage of the sub-data corresponding to the cached mirror data fails, the processor stores the cached mirror data into the memory.

[0108] After selecting the first processor, the network card chip selects another processor (which may be referred to as the second processor) to send the mirror data, and then sends the mirror data to the second processor.

[0109] The network card chip can select the second processor in the following ways:

[0110] Method 1: Pre-establish a processor mapping relationship, where processors are paired. After determining the first processor, the network card chip can search for the corresponding second processor within this mapping relationship. In this mapping relationship, the two corresponding processors are connected to the same memory. For example, in the structure shown in Figure 4, CPU1 and CPU3 are both connected to SSD1. When the network card determines that CPU1 is the first processor, it can further select CPU3 as the second processor and send mirrored data to CPU3.

[0111] Method 2: Randomly select a second processor from other processors besides the first processor. This processing method can be applied to the case where multiple processors are connected to all memories.

[0112] The first processor and the second processor can pre-establish a synchronous cache based on the RDMA protocol. That is, they each apply for a certain amount of cache space in their respective corresponding memories, which can be called the first cache space and the second cache space respectively. Based on the corresponding mechanism of the RDMA protocol, changes in data in one cache space will trigger the same changes in data in the other cache space.

[0113] After receiving the sub-data, the first processor will store the sub-data in the first cache space. After receiving the mirror data corresponding to the sub-data, the second processor will store the mirror data in the second cache space. The first processor stores the sub-data in the corresponding memory. If the storage is successful, the sub-data in the first cache space can be deleted. Then, based on the above mechanism, the deletion of the corresponding mirror data in the second cache space can be triggered. If the storage fails, the mirror data will be stored in the second cache space all the time. When the second processor detects that the cache time of the mirror data reaches the time threshold (the time threshold can be set by the technician based on experience), the second processor can store the mirror data in the memory and delete the mirror data in the second cache space. Then, based on the above mechanism, the deletion of the corresponding sub-data in the first cache space can also be triggered.

[0114] In one possible implementation, the network card chip can send the sub-data and mirrored data to processors on different motherboards. That is, the second processor selected by the network card chip is not on the same motherboard as the first processor. This way, if the motherboard where the first processor resides fails and data cannot be stored in memory, the second processor can ensure that the data can be stored in memory, thereby improving data security.

[0115] The following describes in detail the data storage process for the target device shown in Figure 4. As shown in Figure 7, it includes the following processing steps. Furthermore, Figure 8 illustrates the corresponding transmission process of sub-data, mirror data, and verification data in conjunction with the device structure. Due to the limited content that can be presented in the figure, Figure 8 only shows the data transmission process related to a single data transmission from the network card to a CPU.

[0116] 701. The network card of the target device receives data to be stored sent by other devices and caches the data.

[0117] 702. During the data reception process, when the data in the network card cache reaches 1KB, the 1KB data is used as a portion of data A1, and A1 is copied to obtain mirror data A1'. The network card determines CPU1 among multiple CPUs according to the polling mechanism, and determines CPU3 corresponding to CPU1.

[0118] In the CPU correspondence table, CPU3 corresponding to CPU1 is searched for to process the storage of the corresponding mirror data. In the correspondence table, the two corresponding CPUs belong to different motherboards and are connected to the same SSD. The two CPUs are the main processor and backup processor of the SSD respectively.

[0119] 703, the network card sends A1 to CPU1.

[0120] 704 , CPU1 sends A1 to SSD1 for storage.

[0121] 705 , CPU 1 performs EC calculation on A1 to generate verification data A1”.

[0122] 706, CPU1 sends A1" to CPU2.

[0123] CPU2 is another CPU on the same motherboard as CPU1.

[0124] 707 , CPU 2 sends A1” to SSD 2 for storage.

[0125] There is no necessary order between the above step 704 and steps 705 to 707.

[0126] 708 , the network card sends A1′ to CPU 3 .

[0127] CPU3 caches A1'. If A1 fails to be successfully stored in SSD1, CPU3 sends A1' to SSD1 for storage. If A1 is successfully stored in SSD1, CPU3 deletes the cached A1'.

[0128] There is no necessary order between the above step 708 and steps 703 to 707.

[0129] The network card continues to receive data. When the subsequent received data reaches 1KB again, the above-mentioned processing similar to steps 702-708 can be performed. The difference is that the subsequent other CPUs will be selected according to the polling principle, and the CPUs that send the mirror data and the verification data are also different.

[0130] Optionally, when the network card distributes data received from other devices, it can poll all processors in the target device (for the case of Figure 8, the network card polls CPU1, CPU2, CPU3 and CPU4), or it can poll each processor on the motherboard where the network card is located (for the case of Figure 8, the network card polls CPU1 and CPU2).

[0131] In the embodiment of the present disclosure, the network card divides the received data to be stored and distributes it to multiple processors, which then perform data storage-related processing. Compared to the network card sending all the data to one processor and then distributing it to other processors, the solution of the embodiment of the present disclosure is that the network card distributes the data directly to the processor, without requiring one processor to distribute the received data to other processors. In this way, data transmission between processors can be reduced, and the processors are connected through the inter-chip bus, thereby reducing the occupancy of the inter-chip bus.

[0132] Based on the same technical concept, an embodiment of the present disclosure provides a computer device, as shown in FIG1 , the computer device includes a network card 120 and multiple processors 110 ;

[0133] The network card 120 is configured to receive target data to be stored, divide the received target data into multiple sub-data, and distribute the multiple sub-data to multiple processors 110. Specifically, it can implement the processing functions of steps 501, 502, 701, 702, 703, and 704 above, as well as other implicit steps.

[0134] The processor 110 is configured to: generate corresponding redundant data based on the received sub-data; and store the received sub-data and the corresponding redundant data. Specifically, the processor 110 may implement the processing functions of steps 503, 504, 705, and 706, as well as other implicit steps.

[0135] In a possible implementation, the network card 120 communicates with each processor 110 through different buses. The network card 120 is configured to send the sub-data to the corresponding processor 110 through the bus corresponding to each processor 110 .

[0136] In a possible implementation, the computer device includes multiple mainboards, and at least two processors 110 among the multiple processors 110 are installed on each mainboard.

[0137] In a possible implementation, the number of processors 110 located on each mainboard among the multiple processors 110 is the same.

[0138] In a possible implementation, the network card 120 is configured to divide the received target data into multiple sub-data based on a load balancing mechanism, and distribute the multiple sub-data to each processor 110 .

[0139] In one possible implementation, the computer device includes multiple memories 130, with different processors 110 corresponding to different memories 130. A processor 110 is configured to send received sub-data to a corresponding memory 130 for storage and to send redundant data to at least one other processor 110. The other processors 110 are configured to send received redundant data to a corresponding memory 130 for storage. Specifically, the processing functions of steps 503, 504, 705, and 706 above, as well as other implicit steps, may be implemented.

[0140] In a possible implementation, the processor 110 is configured to send the redundant data to at least one other processor 110 on the same motherboard as the processor 110. Specifically, the processing functions of the above steps 504 and 706 and other implicit steps may be implemented.

[0141] In one possible implementation, the network card 120 is further configured to replicate each sub-data to obtain corresponding mirrored data, and distribute the replicated mirrored data to each processor 110, wherein the mirrored data and the corresponding sub-data are respectively sent to different processors 110. Specifically, the processing functions of steps 601, 602, 702, and 708 above, as well as other implicit steps, may be implemented.

[0142] The processor 110 is further configured to cache the received mirrored data, delete the cached mirrored data when the sub-data corresponding to the cached mirrored data is successfully stored, and store the cached mirrored data in the memory 130 when the sub-data corresponding to the cached mirrored data fails to be stored. Specifically, the processor 110 can implement the processing functions of steps 605 and 708 above, as well as other implicit steps.

[0143] In a possible implementation, the mirror data and the corresponding sub-data are respectively sent to the processors 110 on different mainboards, which can specifically implement the processing functions of the above steps 602 and 708, as well as other implicit steps.

[0144] In the disclosed embodiment, the network card divides the received target data to be stored and distributes it to multiple processors, which then perform data storage-related processing. Compared to the network card sending the entire data to a single processor, which then distributes it to other processors, the disclosed embodiment distributes the data directly to the processors, eliminating the need for one processor to distribute the received data to other processors. This reduces data transmission between processors, which are connected via an inter-chip bus, thereby reducing the occupancy of the inter-chip bus.

[0145] The embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer device can store, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disk (DVD)), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computer device to execute the data storage method.

[0146] The present disclosure also provides a computer program product comprising instructions (or code). The computer program product may be software or a program product comprising instructions that can be run on a computer device or stored in any available medium. When the computer program product is run on at least one computer device, the computer device is caused to execute the data storage method.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A data storage method, characterized in that: The method is applied to a target device, the target device including a network card and multiple processor sets, each of the multiple processor sets including at least one processor, and the method includes: The network card receives target data to be stored; The network card divides the received target data into multiple sub-data, and distributes the multiple sub-data to each processor set; Each processor set generates corresponding redundant data based on the received sub-data; Each processor set stores the received sub-data and corresponding redundant data.

2. The method according to claim 1, characterized in that The network card communicates with each of the processor sets via different buses, wherein the network card distributes the plurality of sub-data to each of the processor sets, specifically including: The network card sends the sub-data to the corresponding processor set through the bus corresponding to each processor set.

3. The method according to claim 1, characterized in that The target device includes a plurality of mainboards, and at least two processor sets of the plurality of processor sets are installed on each mainboard.

4. The method according to claim 3, characterized in that The number of processor sets in the plurality of processor sets located on each mainboard is the same.

5. The method according to any one of claims 1 to 4, characterized in that The network card divides the received target data into a plurality of sub-data, and the network card distributes the plurality of sub-data to each processor set, including: The network card divides the received target data into multiple sub-data based on a load balancing mechanism, and distributes the multiple sub-data to each processor set.

6. The method according to any one of claims 1 to 5, characterized in that The target device includes multiple memories, and different processor sets correspond to different memories; Each processor set stores the received sub-data and corresponding redundant data, including: Each processor set sends the received sub-data to a corresponding memory for storage, and each processor set sends the redundant data to at least one other processor set; Each other processor set sends the received redundant data to the corresponding memory for storage.

7. The method according to claim 6, characterized in that Each processor set sends the redundant data to at least one other processor set, including: Each processor set sends the redundant data to at least one other processor set that belongs to the same motherboard as the processor set.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: The network card copies each sub-data to obtain corresponding mirror data, and distributes the copied mirror data to each processor set, wherein the mirror data and the corresponding sub-data are respectively sent to different processor sets; The processor set caches the received mirror data, deletes the cached mirror data when the sub-data corresponding to the cached mirror data is successfully stored, and stores the cached mirror data to the memory when the sub-data corresponding to the cached mirror data fails to be stored.

9. The method according to claim 8, characterized in that The mirror data and the corresponding sub-data are respectively sent to processor sets on different mainboards.

10. A computer device, characterized in that: The computer device includes a network card and a plurality of processor sets, each of the plurality of processor sets including at least one processor; The network card is configured to receive target data to be stored, divide the received target data into multiple sub-data, and distribute the multiple sub-data to each processor set; The processor set is configured to generate corresponding redundant data based on the received sub-data, and store the received sub-data and the corresponding redundant data.

11. The computer device according to claim 10, wherein: The network card communicates with each processor set via different buses respectively; The network card is used to send the sub-data to the corresponding processor set through the bus corresponding to each processor set.

12. The computer device according to claim 10, wherein: The computer device includes a plurality of mainboards, and at least two processor sets of the plurality of processor sets are mounted on each mainboard.

13. The computer device according to claim 10, wherein: The number of processor sets in the plurality of processor sets located on each mainboard is the same.

14. The computer device according to any one of claims 10 to 13, characterized in that: The network card is used to divide the received target data into multiple sub-data based on a load balancing mechanism, and distribute the multiple sub-data to each processor set.

15. The computer device according to any one of claims 10 to 14, characterized in that: The computer device includes multiple memories, and different processor sets correspond to different memories; Each processor set is configured to send the received sub-data to a corresponding memory for storage, and send the redundant data to at least one other processor set; Each of the other processor sets is configured to send the received redundant data to a corresponding memory for storage.

16. The computer device according to claim 10, wherein: Each processor set is configured to send the redundant data to at least one other processor set that belongs to the same motherboard as the processor set.

17. The computer device according to any one of claims 10 to 16, characterized in that: The network card is further configured to replicate each sub-data to obtain corresponding mirror data, and distribute the replicated mirror data to each processor set, wherein the mirror data and the corresponding sub-data are respectively sent to different processor sets; The processor set is further configured to cache the received mirror data, delete the cached mirror data when the sub-data corresponding to the cached mirror data is successfully stored, and store the cached mirror data in the memory when the sub-data corresponding to the cached mirror data fails to be stored.

18. The computer device according to claim 17, wherein: The mirror data and the corresponding sub-data are respectively sent to processor sets on different mainboards.

19. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program code. When the computer program code is executed by a computer device, the computer device executes the method according to any one of claims 1 to 9.

20. A computer program product, characterized in that The computer program product comprises computer program code. When the computer program code is executed by a computer device, the computer device performs the method according to any one of claims 1 to 9.

Images