Peripheral device interconnection extension device speed limiting system, equipment and cluster

Abstract

The present application provides a peripheral device interconnection extension device speed limiting system, device and cluster. The system includes: a peripheral device interconnection extension device simulated by a data processor, the peripheral device interconnection extension device includes a peripheral device interconnection extension bridge device and a peripheral device interconnection extension terminal device; the peripheral device interconnection extension terminal device has a port level guaranteed limiting rate, the peripheral device interconnection extension terminal device class has a class level guaranteed limiting rate, the peripheral device interconnection extension terminal device group has a group level guaranteed limiting rate, and the peripheral device interconnection extension terminal device has a device level guaranteed limiting rate; the port level guaranteed limiting rate is equal to the sum of the class level guaranteed limiting rates of all peripheral device interconnection extension terminal device classes, the class level guaranteed limiting rate is equal to the sum of all group level guaranteed limiting rates under the class, and the group level guaranteed limiting rate is equal to the sum of all device level guaranteed limiting rates under the group.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a peripheral device interconnection expansion device speed limiting system, device and cluster. Background Art

[0002] The Data Processing Unit (DPU) is a new generation of processors for data-centric computing, integrating complete data center functions into a single chip. With the development of data processors, data processors can simulate more and more peripheral device interconnection and extension end devices, but the traffic between these peripheral device interconnection and extension end devices often competes with each other, causing confusion. Summary of the invention

[0003] The present application provides a peripheral device interconnection extension device speed limiting system, device and cluster, which can provide multiple levels of limit control.

[0004] In a first aspect, a peripheral device interconnection expansion device speed limiting system is provided, comprising:

[0005] A data processor is used to provide a peripheral device interconnection expansion device and a hardware logic of a topological relationship of the peripheral device interconnection expansion device. The data processor is also provided with a simulator. The simulator is used to provide a peripheral device interconnection expansion device and a software logic of a topological relationship of the peripheral device interconnection expansion device, and negotiate and interact with the hardware logic to realize simulation of the peripheral device interconnection expansion device.

[0006] A processor, used to drive the data processor through a driver;

[0007] The peripheral device interconnection expansion device comprises a peripheral device interconnection expansion bridge device and a peripheral device interconnection expansion terminal device, wherein the peripheral device interconnection expansion bridge device is used to connect the peripheral device interconnection expansion terminal device;

[0008] The peripheral device interconnection extension terminal device includes one or more peripheral device interconnection extension terminal device classes, the peripheral device interconnection extension terminal device class includes one or more peripheral device interconnection extension terminal device groups, the peripheral device interconnection extension terminal device group includes one or more peripheral device interconnection extension terminal devices,

[0009] The limiting rate includes a port level limiting rate, a class level limiting rate, a group level limiting rate and a device level limiting rate, wherein the port level limiting rate includes a port level guaranteed limiting rate, the class level limiting rate includes a class level guaranteed limiting rate, the group level limiting rate includes a group level guaranteed limiting rate, and the device level limiting rate includes a device level guaranteed limiting rate;

[0010] Wherein, the peripheral device interconnection extension end device has the port level guaranteed limit rate, the peripheral device interconnection extension end device class has the class level guaranteed limit rate, the peripheral device interconnection extension end device group has the group level guaranteed limit rate, and the peripheral device interconnection extension end device has the device level guaranteed limit rate;

[0011] The port level guaranteed limit rate is equal to the sum of the class level guaranteed limit rates of all peripheral device interconnection extension end device classes, the class level guaranteed limit rate is equal to the sum of all group level guaranteed limit rates under the class, and the group level guaranteed limit rate is equal to the sum of all device level guaranteed limit rates under the group.

[0012] In some possible designs, the port level rate limit also includes a port level peak rate limit, the class level rate limit includes a class level peak rate limit, the group level rate limit includes a group level peak rate limit, and the device level rate limit includes a device level peak rate limit;

[0013] The peripheral device interconnect extension end device has the port level peak rate limit, the peripheral device interconnect extension end device class has the class level peak rate limit, the peripheral device interconnect extension end device group has the group level peak rate limit, and the peripheral device interconnect extension end device has the device level peak rate limit;

[0014] The class-level peak limit rate is less than or equal to the port-level guaranteed limit rate, the group-level peak limit rate is less than or equal to the class-level guaranteed limit rate of the peripheral device interconnection extension end device class to which it belongs, and the device-level peak limit rate is less than or equal to the group-level guaranteed limit rate of the peripheral device interconnection extension end device group to which it belongs.

[0015] In some possible designs, the peripheral device interconnect extension terminal device includes a first peripheral device interconnect extension terminal device class and a second peripheral device interconnect extension terminal device class,

[0016] In the case where the actual traffic of the first peripheral device interconnection extension end device class is less than the class level guaranteed limit rate of the first peripheral device interconnection extension end device class, the excess class level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device class for use, and the actual traffic of the second peripheral device interconnection extension end device class is greater than the class level guaranteed limit rate of the second peripheral device interconnection extension end device class and less than the class level peak limit rate of the second peripheral device interconnection extension end device class.

[0017] In some possible designs, the peripheral device interconnect extension end device class includes a first peripheral device interconnect extension end device group and a second peripheral device interconnect extension end device group.

[0018] In the case where the actual flow of the first peripheral device interconnection extension end device group is less than the group level guaranteed limit rate of the first peripheral device interconnection extension end device group, the excess group level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device group for use, and the actual flow of the second peripheral device interconnection extension end device group is greater than the group level guaranteed limit rate of the second peripheral device interconnection extension end device group and less than the group level peak limit rate of the second peripheral device interconnection extension end device group.

[0019] In some possible designs, the peripheral device interconnection extension terminal device group includes a first peripheral device interconnection extension terminal device and a second peripheral device interconnection extension terminal device.

[0020] In the case where the actual traffic of the first peripheral device interconnection extension end device is less than the device level guaranteed limit rate of the first peripheral device interconnection extension end device, the excess device level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device for use, and the actual traffic of the second peripheral device interconnection extension end device is greater than the device level guaranteed limit rate of the second peripheral device interconnection extension end device and less than the device level peak limit rate of the second peripheral device interconnection extension end device.

[0021] In some possible designs, the peripheral device interconnect extension end device class includes a storage device class, a remote direct memory access device class, and a network device class.

[0022] In some possible designs, peripheral device interconnect extension terminal devices of the same peripheral device interconnect extension terminal device group belong to the same virtual machine.

[0023] In some possible designs, the limiting rate is one or more of a bandwidth limiting rate, a packet limiting rate, and an input and output request processing limiting rate.

[0024] In a second aspect, a computing device is provided, comprising a peripheral device interconnection expansion device speed limiting system and a storage unit, wherein the peripheral device interconnection expansion device speed limiting system and the storage unit can communicate with each other, and the peripheral device interconnection expansion device speed limiting system is the system described in any one of the first aspect.

[0025] In a third aspect, a computing device cluster includes multiple computing devices, at least one computing device includes a peripheral device interconnection expansion device speed limiting system and a storage unit, the peripheral device interconnection expansion device speed limiting system and the storage unit can communicate with each other, and the peripheral device interconnection expansion device speed limiting system is a system as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0027] Figure 1 It is a structural schematic diagram of a peripheral device interconnection expansion device speed limiting system provided by the present application;

[0028] Figure 2 It is a structural diagram of a central processing unit provided by the present application;

[0029] Figure 3 It is a structural diagram of a data processor provided by the present application;

[0030] Figure 4 It is a schematic diagram of a multi-level flow control provided by the present application;

[0031] Figure 5 It is a structural schematic diagram of a computing device provided by this application. DETAILED DESCRIPTION

[0032] The embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention. The terms used in the implementation mode of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0033] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of a peripheral device interconnection expansion device speed limit system provided by this application. Figure 1 As shown, the peripheral device interconnection expansion device speed limiting system provided in the present application includes: a central processing unit (CPU) 11 and a data processing unit (DPU) 12.

[0034] CPU11 is a computing core and a control core, and can handle complex situations. CPU11 can be a very large-scale integrated circuit. The main operating system and other software programs are installed in CPU11, so that CPU11 can access the memory and various PCIE devices. CPU11 can quickly access the local memory through the on-chip bus. CPU11 may include one or more processor cores. In one implementation, CPU11 may be a multi-core chip, that is, a chip containing multiple processing cores. In another implementation, CPU11 may include one or more processor cores. It can be a chip with one processing core.

[0035] See also Figure 2 , Figure 2 Schematic diagram of the structure of a central processing unit provided by this application. Figure 2 As shown, the CPU may include a memory address register 101, a memory data register 102, a program counter 103, an instruction register 104, an instruction decoder 105, a microcode read-only memory 106, an operation controller 107, a calculation unit 108, a general register group 109, an accumulator 110, a program status word register 111, a timing circuit 112, and a processor bus 120. The processor bus 120 may also be a data bus, a power bus, a control bus, or a status signal bus.

[0036] The CPU is used to process instructions and data stored in the memory 113. In some embodiments, the instructions may include one or more instruction formats. The instruction format may indicate various fields (number of bits, position of bits, etc.) to specify the operation of the data processing to be performed and the operand on which the operation will be performed. Some instruction formats may be further defined by instruction templates (or sub-formats).

[0037] In the CPU, the memory address register 101 is used to store the address of the memory that the CPU is currently accessing. The memory data register 102 is used to store the data and read or write instructions that the CPU reads or writes from the address, so as to make up for the difference in operating speed between the central processing unit and the memory.

[0038] The timing circuit 112 provides a time reference for each component through a fixed clock. The time for the central processing unit to execute an instruction is an instruction cycle. The program counter 103 is used to store the address of the next instruction. When the instructions are executed sequentially, the program counter 103 automatically adds the number of bytes of an instruction after each instruction is taken. When a transfer instruction is encountered, the program counter 103 specifies the address of the next instruction through the address code field in the transfer instruction. The instruction register 104 is used to save the instruction currently being executed. The instruction includes two fields, an operation code and an address code. The operation code part is decoded by the instruction decoder 105 to generate the control potential required by the instruction. The operation controller 107 can generate various operation control signals based on the control potential signal output by the instruction decoder and the timing signal generated by the timing circuit 112, and control the remaining components of the CPU to complete the operation of fetching and executing instructions.

[0039] A microinstruction is the smallest unit of instructions executed by a central processing unit. An instruction can be a single microinstruction or composed of several microinstructions. An instruction composed of multiple microinstructions is called a complex instruction. Various mechanisms can be used to implement the decoding of complex instructions by the instruction decoder 105. Specific decoding mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read-only memories (ROMs), and the like. In one embodiment, the steps of the microinstructions corresponding to the complex instructions can be stored in the microcode read-only memory 106. During the decoding process, the instruction decoder 105 can query and obtain the operation code and address code of the microinstructions constituting the complex instructions from the microcode read-only memory 106, and decode the operation code part of the microinstructions in turn to generate the control potential required by the microinstructions.

[0040] The operation controller 107 has multiple buffers, and can send the decoded instructions to their respective reservation stations for storage according to the types of instructions. After analyzing the status of the hardware circuit of the computing unit and the specific situation of whether each instruction can be executed in advance, the instructions that can be executed in advance are scheduled to the corresponding computing unit 108 for execution. During this period, the instruction stream is reordered so that the instruction pipeline can proceed and be smoothly scheduled. Exemplarily, for instructions of integer calculation, the operation controller 107 can use the integer reservation station to save instructions and assign them to the integer computing unit for calculation; for instructions of floating-point calculation, the operation controller 107 can use the floating-point reservation station to save instructions and assign them to the floating-point computing unit for calculation.

[0041] The general register group 109 is used to store the data corresponding to the address code according to the address code of the instruction. The calculation unit 108 is used to receive the operation control signal of the operation controller 107 and perform calculations on the data stored in the general register group 109, including arithmetic operations (including basic operations such as addition, subtraction, multiplication and additional operations) and logical operations (including shifting, logical testing or comparison of two values). The temporary variables generated in the calculation are stored in the accumulator 110, and the information of the generated status is stored in the program status word register 111, such as the operation result carry / borrow flag (C), the operation result overflow flag (O), the operation result zero flag (Z), the operation result negative flag (N), the operation result sign flag (S), etc. The program status word register is also used to save information such as interrupts and the working status of the computing device, so that the CPU can timely understand the machine operation status and program operation status.

[0042] The calculation unit 108 includes a variety of different circuit modules, which can be used to execute different instructions. For example, the integer calculation unit 1081 and the floating point calculation unit 1082 are used to perform arithmetic operations and logical operations on integers and floating point numbers, respectively.

[0043] It can be understood that the above-mentioned CPU is only a specific embodiment. In actual applications, it may have more or fewer components, which will not be described in detail here.

[0044] The DPU 12 includes multiple processing cores, which can process data in parallel. Therefore, the DPU 12 often has powerful computing capabilities. Figure 3, the DPU includes: multiple memories 121, multiple processing cores 122, and communication interfaces 123 and other hardware resources. For example, the processing core 122 may include a first processing core, a second processing core, ..., an Nth processing core, and the memory 121 includes a first memory, a second memory, ..., an Nth memory. In a possible embodiment, the computing power of the processing core 122 in the DPU12 is weaker than the computing power of the CPU11, and the storage capacity of the memory 121 in the DPU12 is weaker than the storage capacity of the memory in the CPU11. Therefore, the CPU11 is suitable for processing a small number of complex instructions, while the DPU12 is suitable for processing batches of simple instructions. The memory 121 may be a non-volatile solid-state memory. In a possible embodiment, the memory 121 includes a read-only memory ROM (Read-Only Memory). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory, or a combination of two or more of these. A slave operating system may be installed on the DPU, and the slave operating system is used to manage the hardware resources on the DPU. In a possible embodiment, the function of the slave operating system in the DPU12 is weaker than the main operating system in the CPU11. The DPU12 may be used to simulate the topological relationship between at least one peripheral device interconnection expansion device and at least one PCIE device. The hardware resources of the DPU are used to provide the hardware logic of the topological relationship of the PCIE device and the PCIE device. A simulator may be installed in the DPU12, and the simulator is used to provide the software logic of the topological relationship of the PCIE device and the PCIE device. The DPU12 implements the simulation of the PCIE device through negotiation and interaction between the software logic and the hardware logic.

[0045] The PCIE devices simulated in the DPU may include PCIE end devices and PCIE bridge devices. PCIE end devices can provide various functions. For example, the PCIE end device can be a network card that provides network data transmission functions, or a sound card that provides sound source processing. The PCIE bridge device is used to connect multiple PCIE end devices to form different topological relationships. Specifically, the PCIE bridge device includes multiple end device slots, and each end device slot is used to mount a peripheral device interconnection expansion end device. The topological relationship of the PCIE device may include one or more of a star relationship, a tree relationship, a ring relationship, a bus relationship, etc. Figure 1Taking the topological relationship of the PCIE devices as an example, the PCIE bridge device serves as the root node of the tree, and multiple PCIE end devices are respectively connected to the root node PCIE bridge device.

[0046] PCIE end devices may include one or more of network devices, Remote Direct Memory Access (RDMA) devices, storage devices, etc. Among them, PCIE end devices can be organized in the following manner: all PCIE end devices simulated in the DPU are organized as port levels, and the same type of PCIE end devices are organized as the same class level, wherein the same class level refers to the same network device, or the same RDMA device, or the same storage device.

[0047] The rate limits of PCIE end devices include port-level rate limits, class-level rate limits, group-level rate limits, and device-level rate limits. Among them, the port-level rate limits are the rate limits for all PCIE end devices simulated by the DPU. The class-level rate limits are the rate limits for PCIE end devices of the same class. For example, storage devices are PCIE end devices of the same class, RDMA devices are PCIE end devices of the same class, and network devices are PCIE end devices of the same class. The group-level rate limits are the rate limits for PCIE end devices of the same group under the same class of PCIE end devices. For example, storage devices belonging to the same virtual machine are PCIE end devices of the same group. The device-level rate limits are the rate limits for a certain PCIE end device.

[0048] The rate limit includes one or more of the guaranteed rate limit and the peak rate limit. The guaranteed rate limit is used to specify a minimum rate guarantee to ensure that the network or system can maintain this minimum level of service quality under any circumstances. The peak rate limit is used to set a maximum rate limit to control the amount of data transmission or request allowed in a short period of time to prevent exceeding the system's processing capacity.

[0049] The rate limiting is one or more of bandwidth rate limiting, packet rate limiting, and input / output request processing rate limiting. Bandwidth throttling refers to limiting or controlling the network transmission rate to ensure fair use of network resources, optimize network performance, or meet specific business requirements. Bandwidth is the amount of data transmitted per unit time in a network, usually expressed in bits per second (bps), kilobits per second (kbps), megabits per second (Mbps), or gigabits per second (Gbps). Packet rate limiting refers to the technology of controlling and limiting the number of packets transmitted in a network. This is different from bandwidth throttling, which usually focuses on the rate of data transmission (i.e., the amount of data), while packet rate limiting focuses on the number of packets transmitted per unit time. A packet is the basic unit of transmission in a network, containing data and its control information (such as source address, destination address, sequence number, etc.). I / O request rate limiting involves limiting the request frequency of input and output operations in the system to optimize system performance and resource usage. Among them, input and output (I / O) requests include disk reading and writing, network data transmission and other requests.

[0050] Therefore, the port level limiting rate includes one or more of the port level bandwidth limiting rate, the port level packet limiting rate, and the port level input and output request processing limiting rate. More specifically, the port level bandwidth limiting rate includes the port level bandwidth guarantee limiting rate and the port level bandwidth peak limiting rate. The port level packet limiting rate includes the port level packet guarantee limiting rate and the port level packet peak limiting rate. The port level input and output request processing limiting rate includes the port level input and output request processing guarantee limiting rate and the port level input and output request processing peak limiting rate. Similarly, the class level limiting rate, the group level limiting rate, and the device level limiting rate are also similar and are not specifically limited here.

[0051] The port-level guaranteed limit rate is equal to the sum of the class-level guaranteed limit rates of all peripheral device interconnect extension end device classes. The class-level guaranteed limit rate is equal to the sum of all group-level guaranteed limit rates under the class. The group-level guaranteed limit rate is equal to the sum of all device-level guaranteed limit rates under the group. The device-level guaranteed limit rate is the guaranteed rate of the device.

[0052] The class-level peak limit rate is less than or equal to the port-level guaranteed limit rate, the group-level peak limit rate is less than or equal to the class-level guaranteed limit rate of the peripheral device interconnect extension end device class to which it belongs, and the device-level peak limit rate is less than or equal to the group-level guaranteed limit rate of the peripheral device interconnect extension end device group to which it belongs.

[0053] When the peripheral device interconnection extension end device includes a first peripheral device interconnection extension end device class and a second peripheral device interconnection extension end device class, when the actual traffic of the first peripheral device interconnection extension end device class is less than the class level guaranteed limit rate of the first peripheral device interconnection extension end device class, the excess class level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device class for use, and the actual traffic of the second peripheral device interconnection extension end device class is greater than the class level guaranteed limit rate of the second peripheral device interconnection extension end device class and less than the class level peak limit rate of the second peripheral device interconnection extension end device class.

[0054] When the peripheral device interconnection extension end device class includes a first peripheral device interconnection extension end device group and a second peripheral device interconnection extension end device group, in a case where the actual flow of the first peripheral device interconnection extension end device group is less than the group level guaranteed limit rate of the first peripheral device interconnection extension end device group, the excess group level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device group for use, and the actual flow of the second peripheral device interconnection extension end device group is greater than the group level guaranteed limit rate of the second peripheral device interconnection extension end device group and less than the group level peak limit rate of the second peripheral device interconnection extension end device group.

[0055] When the peripheral device interconnection extension end device group includes a first peripheral device interconnection extension end device and a second peripheral device interconnection extension end device, in a case where the actual flow of the first peripheral device interconnection extension end device is less than the device level guaranteed limit rate of the first peripheral device interconnection extension end device, the excess device level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device for use, and the actual flow of the second peripheral device interconnection extension end device is greater than the device level guaranteed limit rate of the second peripheral device interconnection extension end device and less than the device level peak limit rate of the second peripheral device interconnection extension end device.

[0056] For example, Figure 4As shown, it is assumed that the PCIE end device includes a first network device, a second network device, a third network device, a first RDMA device, a second RDMA device, a third RDMA device, a first storage device, a second storage device, a third storage device, and the like. The first network device and the second network device constitute a first network device group, and the third network device constitutes a second network device group. The first RDMA device constitutes a first RDMA device group, the second RDMA device and the third RDMA device constitute a second RDMA device group, and the first storage device, the second storage device and the third storage device constitute a first storage device group. The first network device group and the second network device group constitute a first PCIE end device class, the first RDMA device group and the second RDMA device group constitute a second PCIE end device class, the first storage device group constitutes a third PCIE end device class, and the first PCIE end device class, the second PCIE end device class and the third PCIE end device class constitute a port.

[0057] Among them, the first network device has a first network device level guaranteed limit rate and a first network device level peak limit rate, the second network device has a second network device level guaranteed limit rate and a second network device level peak limit rate, and the third network device has a third network device level guaranteed limit rate and a third network device level peak limit rate. The first RDMA device has a first RDMA device level guaranteed limit rate and a first RDMA device level peak limit rate, the second RDMA device has a second RDMA device level guaranteed limit rate and a second RDMA device level peak limit rate, and the third RDMA device has a third RDMA device level guaranteed limit rate and a third RDMA device level peak limit rate. The first storage device has a first storage device level guaranteed limit rate and a first storage device level peak limit rate, the second storage device has a second storage device level guaranteed limit rate and a second storage device level peak limit rate, and the third storage device has a third storage device level guaranteed limit rate and a third storage device level peak limit rate. The first network device group has a first network group level guaranteed limit rate and a first network group level peak limit rate. The second network device group has a second network group level guaranteed limit rate and a second network group level peak limit rate. The first RDMA device group has a first RDMA group level guaranteed limit rate and a first RDMA group level peak limit rate. The second RDMA device group has a second RDMA group level guaranteed limit rate and a second RDMA group level peak limit rate. The first storage device group has a first storage group level guaranteed limit rate and a first storage group level peak limit rate. The first PCIE end device class has a first PCIE end device class level guaranteed limit rate and a first PCIE end device class level peak limit rate. The second PCIE end device class has a second PCIE end device class level guaranteed limit rate and a second PCIE end device class level peak limit rate. The third PCIE end device class has a third PCIE end device class level guaranteed limit rate and a third PCIE end device class level peak limit rate. The port has a port level guaranteed limit rate and a port level peak limit rate.

[0058] When the actual flow of the first network device is less than the device level guaranteed limit rate of the first network device, the excess device level guaranteed limit rate is allocated to the second network device for use. Here, the actual flow of the second network device can be greater than the device level guaranteed limit rate of the second network device and less than the device level peak limit rate of the second network device.

[0059] When the actual flow of the second RDMA device is less than the device level guaranteed limit rate of the second RDMA device, the excess device level guaranteed limit rate is allocated to the third RDMA device for use. Here, the actual flow of the third RDMA device may be greater than the device level guaranteed limit rate of the third RDMA device and less than the device level peak limit rate of the third RDMA device.

[0060] When the actual flow of the first storage device is less than the device level guaranteed limit rate of the first storage device, and the actual flow of the second storage device is less than the device level guaranteed limit rate of the second storage device, the excess device level guaranteed limit rate is allocated to the third storage device for use. Here, the actual flow of the third storage device may be greater than the device level guaranteed limit rate of the third storage device, and less than the device level peak limit rate of the third storage device.

[0061] When the actual flow of the first network device group is less than the group-level guaranteed limit rate of the first network device group, the excess group-level guaranteed limit rate is allocated to the second network device group for use. Here, the actual flow of the second network device group can be greater than the group-level guaranteed limit rate of the second network device group and less than the group-level peak limit rate of the second network device group.

[0062] In the case where the actual flow of the first RDMA device group is less than the group-level guaranteed limit rate of the first RDMA device group, the excess group-level guaranteed limit rate is allocated to the second RDMA device group for use. Here, the actual flow of the second RDMA device group may be greater than the group-level guaranteed limit rate of the second RDMA device group and less than the group-level peak limit rate of the second RDMA device group.

[0063] When the actual flow of the first PCIE end device class is less than the class level guaranteed limit rate of the first PCIE end device class, and the actual flow of the second PCIE end device class is less than the class level guaranteed limit rate of the second PCIE end device class, the redundant class level guaranteed limit rate is allocated to the third PCIE end device class for use. Here, the actual flow of the third PCIE end device class may be greater than the class level guaranteed limit rate of the third PCIE end device class, and less than the class level peak limit rate of the third PCIE end device class.

[0064] Since the entire PCIE end device class in the DPU is simulated by the DPU, the traffic can be freely allocated between network devices, RDMA devices and storage devices.

[0065] It can be understood that the above example is based on the example of setting a rate limit for each device and each layer. In actual applications, rate limits can be set for some devices, some groups, and some classes as needed, and no specific limitation is made here.

[0066] Taking the bandwidth limit rate as an example, assume that the port-level bandwidth limit rate is 100 Gigabits (G). The port-level bandwidth limit rate is divided into the network class-level bandwidth limit rate, the RDMA class-level bandwidth limit rate, and the storage class-level bandwidth limit rate. Among them, the network class-level bandwidth guarantee limit rate is configured as 40G, and the network class-level bandwidth peak limit rate is configured as 40G. Among the RDMA class-level bandwidth limit rate, the RDMA class-level bandwidth guarantee limit rate is configured as 40G, and the RDMA class-level bandwidth peak limit rate is configured as 40G. Among the storage class-level bandwidth limit rate, the storage class-level bandwidth guarantee limit rate is configured as 20G, and the storage class-level bandwidth peak limit rate is configured as 20G. The network class-level bandwidth guarantee limit rate is divided into the first network device group and the second network device group. Among them, the group-level bandwidth guarantee limit rate of the first network device group is configured as 30G, and the group-level bandwidth peak limit rate of the first network device group is configured as 30G. The group-level bandwidth guarantee limit rate of the second network device group is configured as 10G, and the group-level bandwidth peak limit rate of the second network device group is configured as 10G. Similarly, the RDMA class-level bandwidth guarantee limit rate is allocated to the first RDMA device group and the second RDMA device group. Among them, the group-level bandwidth guarantee limit rate of the first RDMA device group is configured to 20G, and the group-level bandwidth peak limit rate of the first RDMA device group is configured to 20G. The group-level bandwidth guarantee limit rate of the second RDMA device group is configured to 20G, and the group-level bandwidth peak limit rate of the second RDMA device group is configured to 20G. The group-level bandwidth guarantee limit rate of the first storage device group is configured to 20G, and the class-level bandwidth peak limit rate of the first RDMA device group is configured to 20G.

[0067] The group-level bandwidth guarantee limit rate of the first network device group is allocated to the first network device and the second network device. The device-level bandwidth guarantee limit rate of the first network device is configured as 20G, and the device-level bandwidth peak limit rate of the first network device is configured as 30G. The device-level bandwidth guarantee limit rate of the second network device is configured as 10G, and the device-level bandwidth peak limit rate of the second network device is configured as 30G. The group-level bandwidth guarantee limit rate of the second network device group is allocated to the third network device. The device-level bandwidth guarantee limit rate of the third network device is configured as 10G, and the device-level bandwidth peak limit rate of the third network device is configured as 10G.

[0068] The group-level bandwidth guarantee limit rate of the first RDMA device group is allocated to the first RDMA device. The device-level bandwidth guarantee limit rate of the first RDMA device is configured to 20G, and the device-level bandwidth peak limit rate of the first RDMA device is configured to 20G. The group-level bandwidth guarantee limit rate of the second RDMA device group is allocated to the second RDMA device and the third RDMA device. The device-level bandwidth guarantee limit rate of the second RDMA device is configured to 10G, and the device-level bandwidth peak limit rate of the second RDMA device is configured to 20G. The device-level bandwidth guarantee limit rate of the third RDMA device is configured to 10G, and the device-level bandwidth peak limit rate of the third RDMA device is configured to 20G.

[0069] The group-level bandwidth guarantee limit rate of the first storage device group is allocated to the first storage device, the second storage device, and the third storage device. The device-level bandwidth guarantee limit rate of the first storage device is configured to 10G, and the device-level bandwidth peak limit rate of the first storage device is configured to 20G. The device-level bandwidth guarantee limit rate of the second storage device is configured to 5G, and the device-level bandwidth peak limit rate of the second storage device is configured to 20G. The device-level bandwidth guarantee limit rate of the third storage device is configured to 5G, and the device-level bandwidth peak limit rate of the third storage device is configured to 20G.

[0070] The above class-level speed limit configuration ensures that the limit rate is equal to the peak limit rate, ensuring that the traffic of a certain class will not preempt the traffic of other classes. For example, the maximum traffic occupied by the network class is 40G, which will not preempt the 40G traffic of the RDMA class and the 20G traffic of the storage class.

[0071] The above group-level speed limit configuration ensures that the limit rate is equal to the peak limit rate, ensuring that a virtual machine will not preempt the traffic of other virtual machines. For example, the maximum traffic occupied by the virtual machine corresponding to the first network device group is 30G, which will not preempt the traffic 10G of the virtual machine corresponding to the second network device group.

[0072] The above device-level speed limit configures the guaranteed limit rate of the device to be less than the peak limit rate of the device, and the peak limit rate of the device is equal to the guaranteed limit rate at the group level. It ensures that the guaranteed limit rates configured for all devices in the group are guaranteed, and when a device in the group is not used or uses less than the guaranteed limit rate of the device, the remaining traffic can be distributed to other devices in the same group without affecting the business using the device. For example, the guaranteed limit rates of 20G and 10G of the first network device and the second network device of the virtual machine corresponding to the first network device group can be guaranteed. If the first network device only uses the guaranteed limit rate of 5G at a certain moment, the remaining 15G of the guaranteed limit rate can be used by the second network device under the same virtual machine, and the actual traffic of the second network device will become 10G+15G=25G. Because 25G is less than the peak limit rate of the device, it can be used normally.

[0073] It can be understood that the above examples are only specific examples. In actual applications, the rate limits at different levels may also be other values, etc., which are not specifically limited here.

[0074] See also Figure 5 , Figure 5 4 is a schematic diagram of a computing device provided by the present application. The computing device 400 includes: one or more processing units 410 , a communication interface 420 and a storage unit 430 .

[0075] The processing unit 410, the communication interface 420 and the storage unit 430 are interconnected via a bus 440. Optionally, the computing device 400 may also include an input / output interface 450, which is connected to an input / output device for receiving parameters set by a user, etc. The computing device 400 can be used to implement some or all of the functions of the device embodiment or system embodiment in the above-mentioned embodiment of the present application; the processing unit 410 can also be used to implement some or all of the operation steps of the method embodiment in the above-mentioned embodiment of the present application. For example, the specific implementation of the various operations performed by the computing device 400 can refer to the specific details in the above-mentioned embodiment, such as the processing unit 410 is used to perform some or all of the steps in the above-mentioned method embodiment or some or all of the operations in the above-mentioned method embodiment. For another example, in the embodiment of the present application, the computing device 400 can be used to implement some or all of the functions of one or more components in the above-mentioned device embodiment, and the communication interface 420 can be specifically used for the communication functions necessary to implement the functions of these devices and components, and the processing unit 410 can be specifically used for the processing functions necessary to implement the functions of these devices and components, etc.

[0076] Figure 5The computing device 400 may include one or more processing units 410, and the multiple processing units 410 may provide processing capabilities in a coordinated manner in a parallel connection manner, a serial connection manner, a serial-parallel connection manner, or any connection manner, or the multiple processing units 410 may constitute a processor sequence or a processor array, or the multiple processing units 410 may be divided into a main processor and an auxiliary processor, or the multiple processing units 410 may have different architectures such as a heterogeneous computing architecture. In addition, Figure 5 The computing device 400 shown in the figure, and the related structural description and functional description are exemplary and non-limiting. In some exemplary embodiments, the computing device 400 may include Figure 5 More or fewer components may be shown, or some components may be combined or separated, or may have a different arrangement of components.

[0077] The processing unit 410 may have a variety of specific implementation forms. For example, the processing unit 410 may include a central processing unit (CPU), a graphic processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU) or a data processing unit (DPU), etc., and the embodiment of the present application does not specifically limit it. The processing unit 410 may also be a single-core processor or a multi-core processor. The processing unit 410 may be a combination of a CPU and a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. The processing unit 410 may also be implemented using a logic device with built-in processing logic alone, such as an FPGA or a digital signal processor (DSP). The communication interface 420 may be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface may be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface may be a cellular network interface or a wireless local area network interface, etc.

[0078] The storage unit 430 may be a non-volatile memory, such as a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The storage unit 430 may also be a volatile memory, which may be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). The storage unit 430 may also be used to store program codes and data, so that the processing unit 410 can call the program codes stored in the storage unit 430 to execute some or all of the operation steps in the above method embodiment, or to execute the corresponding functions in the above device embodiment. Figure 5 Show more or fewer components, or have different component configurations.

[0079] The bus 440 may be a peripheral component interconnect express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 440 may be divided into an address bus, a data bus, a control bus, etc. In addition to the data bus, the bus 440 may also include a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0080] The present application also provides a system, which includes multiple computing devices, and the structure of each computing device can refer to the structure of the computing device described above. The functions or operations that can be implemented by the system can refer to the specific implementation steps in the above method embodiment and / or the specific functions described in the above device embodiment, which will not be repeated here.

[0081] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one network site, computer, server or data center to another network site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or it can be a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape, etc.), an optical medium (e.g., a DVD, etc.), or a semiconductor medium (e.g., a solid-state hard disk), etc. In the above embodiments, the description of each embodiment has different emphases. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

Claims

1. A peripheral device interconnection expansion device speed limiting system, characterized in that: include: A data processor is used to provide a peripheral device interconnection expansion device and a hardware logic of a topological relationship of the peripheral device interconnection expansion device. The data processor is also provided with a simulator. The simulator is used to provide a peripheral device interconnection expansion device and a software logic of a topological relationship of the peripheral device interconnection expansion device, and negotiate and interact with the hardware logic to realize simulation of the peripheral device interconnection expansion device. A processor, used to drive the data processor through a driver; The peripheral device interconnection expansion device comprises a peripheral device interconnection expansion bridge device and a peripheral device interconnection expansion terminal device, wherein the peripheral device interconnection expansion bridge device is used to connect the peripheral device interconnection expansion terminal device; The peripheral device interconnection extension terminal device includes one or more peripheral device interconnection extension terminal device classes, the peripheral device interconnection extension terminal device class includes one or more peripheral device interconnection extension terminal device groups, the peripheral device interconnection extension terminal device group includes one or more peripheral device interconnection extension terminal devices, The limiting rate includes a port level limiting rate, a class level limiting rate, a group level limiting rate and a device level limiting rate, wherein the port level limiting rate includes a port level guaranteed limiting rate, the class level limiting rate includes a class level guaranteed limiting rate, the group level limiting rate includes a group level guaranteed limiting rate, and the device level limiting rate includes a device level guaranteed limiting rate; Wherein, the peripheral device interconnection extension end device has the port level guaranteed limit rate, the peripheral device interconnection extension end device class has the class level guaranteed limit rate, the peripheral device interconnection extension end device group has the group level guaranteed limit rate, and the peripheral device interconnection extension end device has the device level guaranteed limit rate; The port level guaranteed limit rate is equal to the sum of the class level guaranteed limit rates of all peripheral device interconnection extension end device classes, the class level guaranteed limit rate is equal to the sum of all group level guaranteed limit rates under the class, and the group level guaranteed limit rate is equal to the sum of all device level guaranteed limit rates under the group.

2. The system according to claim 1, characterized in that The port level rate limit also includes a port level peak rate limit, the class level rate limit includes a class level peak rate limit, the group level rate limit includes a group level peak rate limit, and the device level rate limit includes a device level peak rate limit; The peripheral device interconnect extension end device has the port level peak rate limit, the peripheral device interconnect extension end device class has the class level peak rate limit, the peripheral device interconnect extension end device group has the group level peak rate limit, and the peripheral device interconnect extension end device has the device level peak rate limit; The class-level peak limit rate is less than or equal to the port-level guaranteed limit rate, the group-level peak limit rate is less than or equal to the class-level guaranteed limit rate of the peripheral device interconnection extension end device class to which it belongs, and the device-level peak limit rate is less than or equal to the group-level guaranteed limit rate of the peripheral device interconnection extension end device group to which it belongs.

3. The system according to claim 2, characterized in that The peripheral device interconnection extension terminal device includes a first peripheral device interconnection extension terminal device class and a second peripheral device interconnection extension terminal device class, In the case where the actual traffic of the first peripheral device interconnection extension end device class is less than the class level guaranteed limit rate of the first peripheral device interconnection extension end device class, the excess class level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device class for use, and the actual traffic of the second peripheral device interconnection extension end device class is greater than the class level guaranteed limit rate of the second peripheral device interconnection extension end device class and less than the class level peak limit rate of the second peripheral device interconnection extension end device class.

4. The system according to claim 2, characterized in that The peripheral device interconnection extension terminal device class includes a first peripheral device interconnection extension terminal device group and a second peripheral device interconnection extension terminal device group, In the case where the actual flow of the first peripheral device interconnection extension end device group is less than the group level guaranteed limit rate of the first peripheral device interconnection extension end device group, the excess group level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device group for use, and the actual flow of the second peripheral device interconnection extension end device group is greater than the group level guaranteed limit rate of the second peripheral device interconnection extension end device group and less than the group level peak limit rate of the second peripheral device interconnection extension end device group.

5. The system according to claim 2, characterized in that The peripheral device interconnection extension terminal device group includes a first peripheral device interconnection extension terminal device and a second peripheral device interconnection extension terminal device, In the case where the actual traffic of the first peripheral device interconnection extension end device is less than the device level guaranteed limit rate of the first peripheral device interconnection extension end device, the excess device level guaranteed limit rate is allocated to the second peripheral device interconnection extension end device for use, and the actual traffic of the second peripheral device interconnection extension end device is greater than the device level guaranteed limit rate of the second peripheral device interconnection extension end device and less than the device level peak limit rate of the second peripheral device interconnection extension end device.

6. The system according to any one of claims 1 to 5, characterized in that: The peripheral device interconnection expansion terminal device class includes a storage device class, a remote direct memory access device class and a network device class.

7. The system according to any one of claims 1 to 5, characterized in that: The peripheral device interconnect extension terminal devices of the same peripheral device interconnect extension terminal device group belong to the same virtual machine.

8. The system according to any one of claims 1 to 5, characterized in that: The limiting rate is one or more of a bandwidth limiting rate, a packet limiting rate, and an input / output request processing limiting rate.

9. A computing device, characterized in that It comprises a peripheral device interconnection expansion device speed limiting system and a storage unit, wherein the peripheral device interconnection expansion device speed limiting system and the storage unit can communicate with each other, and the peripheral device interconnection expansion device speed limiting system is a system as claimed in any one of claims 1 to 8.

10. A computing device cluster, characterized in that: It comprises a plurality of computing devices, at least one computing device comprises a peripheral device interconnection expansion device speed limiting system and a storage unit, the peripheral device interconnection expansion device speed limiting system and the storage unit can communicate with each other, and the peripheral device interconnection expansion device speed limiting system is a system as described in any one of claims 1 to 8.

Images